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EXECUTIVE SUMMARY 


This.research was motivated by the desire to further understanding 
on how artificial intelligence (AI) systems may effectively support 6 

practitioners engaged in fault management in dynamic situations. One 
standard approach for diagnostic assistance is to provide retrospective 
explanations, but these are not well suited to the demands of dynamic fault 
management. Such explanations occur as interruptions in the flow of work 
and result in data overload. A field study of human practitioners in one 
dynamic fault management application (anesthesiologists during 
ne urosur gi ca l operations) was undertaken in order to gain insight into 
effective diagnostic support among team members. The conceptual 
ramework that guided the field study drew from research on cooperation in 
communication, and particularly on work from conversation analysis on the 
common ground" maintained during coordinative activity. 

The findings indicate that team members assist one another in 
maintaining accurate interpretations of the process by helping one another 
keep track of influences on the process. Two ways they do this are by 
providing unprompted reports of their relevant activities on the process and 
y providing informative responses that go beyond answering explicitly posed 
questions. Episodes of management and diagnosis show that causal 
explanations among team members are better described as joint 
interpretations (in which both team members are involved in the process of 
attaining a mutual interpretation), rather than as retrospective explanations 
given from one team member to another. Explanations of assessments and 
activities that are found are typically brief and embedded in the flow of 
activity. 


The general implications for the design of intelligent systems 
intended to support practitioners in dynamic fault management are that such 
systems should not be "dark boards" concerning their activities and 
assessments. But, because they lack many of the sophisticated capabilities 
isp ayed in human communication, intelligent system design must avoid 
distracting their human partners in an effort to maintain the common 
ground. Instead, the focus should be on providing intelligent system 
assessments and information about activities in the context of (i.e., relative 
to) events in the dynamic process. 


vi 
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CHAPTER I 

INTRODUCTION AND OVERVIEW 


Introduction 

Certain fields of practice involve the management and control of 
complex dynamic systems. These include flight deck operations in 
commercial aviation, control of space systems, anesthetic management 
during surgery or chemical or nuclear process control. Fault diagnosis of these 
dynamic systems generally must occur with the monitored process* on-line 
and in conjunction with maintaining system integrity. 

Some of the demands of fault management in a dynamic process 
include: the need to form interpretations of the situation with faulty data or 
before all the data are available, the need to continuously update these 
interpretations as data comes in or is changed, and the need to act based on 
these interpretations (in order to prevent possible dire consequences or to 
find out more about possible faults, e.g.. Woods, 1994). Artificial intelligence 
( ) and automation is increasingly being used to assist practitioners who 

manage and control complex dynamic systems. One common role for AI 
systems is to function as advisors, presenting diagnoses and 
recommendations. In addition they may also function as intelligent 
subordinate agents taking actions on the monitored process, but supervised to 
some extent by humans. The nature of the interaction (or support paradigm) 
between humans and intelligent system (IS) is critical for the success of the 
joint human-intelligent system. 

One problem with standard forms of assistance in dynamic fault 
management is a lack of coordination between AI explanations and the 
demands of evolving, diagnostic situations (Malin et al., 1991). Poor design of 
the interaction and coordination between automation and human have been 
™P hcated * n P r °bl ems * n managing automated resources and in failures 
(Billings, 1991; Norman, 1990). In particular, forms of interaction that take the 
human out of the problem solving loop, while still leaving the person 
responsible for the problem's solution have been shown to be ineffective 
(Roth, Bennett and Woods, 1987; Billings, 1991). There is a need to create 
more cooperative intelligent systems- systems that assist the human 
practitioner in the process of problem solving and decision making, rather 


and ma " aged as the ' monl,ored * 
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1994) S 2 mply pr ° Viding a solution or recommendation (e.g., Layton et al., 

. . in Th f S research seeks t0 understand in more detail what it means for an 
ntelligent system to function cooperatively, or as a "team player" in complex 
dynamic environments. The approach taken was to study huLm P ' 

L aC l , rS engaged m the management of a complex, dynamic process- 
nesthesiologists during neurosurgical operations. The investigation focused 
nderstanding how team members cooperate in management and fault 
diagnosis and comparing this interaction to the situation with an AI system 
that provides diagnoses and explanations. Of particular concern was to^tudv 
he w a y S m which practitioners support one another in keeping Tware of 

h! L T 1 ™ concernin g the state of the monitoredprocess and of 
the problem solving process. F ana ot 

An Illustrative Example 

In the expert system model for problem solving assistance the AI 

ys em provides diagnoses and/ or recommendations, typically accompanied 

" t?™ pl T tl ° n f ° r the dia « n0sis or commendation. The 

Spknatto/ thaVrelT' t* 150 V”* °‘ ex P lanation ; « * usually a Stsal 
explanahoii that relates observed symptoms to some underlying cause(s). The 

p anation of the diagnosis, on the other hand, is generally a description of 

w he s y st em arnved at the diagnosis. Chandrasekaran, Tanner and 

Josephson (1989) refer to this distinction as "explanations of the world" and 

explanations of decisions" respectively.3 This distinction will be relevant 

throughout this report. We will often refer to the first as interpretations 

exnllZ! han diag ™ ses ' which has a connotation of completenes^s), and 
explanations of interpretations (or simply, explanations). 

o motivate discussion of the problems with the standard expert 

ihP f Ti f ° rm ° f diagI } ostic ex P lana tion for dynamic fault management we use 
the following example. Though the example is hypothetical, the 

c aractenstra and cognitive implications are based on actual systems (see 
Woods, 1988; Malin et al., 1991; Woods, 1994). 7 

Imagine that you are able to observe a practitioner at work on 

™? ag ^ ng S0 ™ e P rocess - F or illustrative purposes, you will also have access 

it mmhf h^h COg 7 hVe activities - What process is, doesn't really matter- 
t might be chemical, nuclear, or even a physiological process like a patient 

undergoing surgery. The important aspects of the situation are that The 
process is dynamic, that it is important to avoid certain states, and that the 

to emphasize the person's role as 
for example. Chandn,**,,,,, « al. 

system's compiled knowledge by linking it to d^Diro h wl^r^Tww n i,T re or . were ™ t made, 2) justifying 
control behavior and problem saving stately P fr ° m wluch rt was denved ^ and 3) explaining 8 
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process cannot be taken off-line while diagnosis proceeds. Information about 
the state of the process is available via a monitor that displays raw parameter 
values. There is also an expert system, which receives data directly from the 
process, and provides assessments on another monitor to the practitioner. 
The expert system's main display has three main windows that provide: 
status messages, diagnoses and explanations of the diagnoses. Now, consider 
the following hypothetical scenario. 

The practitioner has been monitoring the process display when he 
notices an unexpected anomaly in a process parameter. Because the 
anomaly (deviation from a standard condition) is unexpected, he 4 begins 
diagnosis-- thinking about what fault or faults could account for the 
anomaly. In order to better assess the situation, he calls up a few other 
windows on the process display that contain some relevant parameter 
values. From the raw data in the different windows he pieces together a 
picture of what may be going on- two hypotheses jump to mind. He 
glances over to the expert system but its diagnosis window is blank. 
Shifting attention back to the process display, he notices that the initially 
anomalous parameter is continuing its trend. He also notices that another 
parameter is anomalous. He quickly considers the safing actions^ that 
should be taken, and then goes back to the expert system display and 
retrieves the automated subsystem command window. He gives 
instructions for it to execute the safing action. He turns back to the process 
display in order to evaluate the action on the process. 

The safing action has both therapeutic and diagnostic value. Seeing its 
effect on the process leads the practitioner to investigate another set of 
parameter values. In order to see potential patterns, he needs to call up 
several windows at once. As he begins to do this, he notices an anomalous 
trend; another hypothesis springs to mind-one with more severe 
consequences than the two that initially occurred to him. He searches 
through the interface to gather more information. Suddenly the expert 
system display beeps. He makes a note to look at it as soon as he finds the 
relevant data he is looking for. When he does, he looks away from the 
process data screen to see what hypothesis has been posted in the 
diagnostic window. It is one of the lower priority ones he initially 
considered. He re-checks the current parameter values in the process 
display. He wonders whether the expert system has taken into account the 
other, more dire, possibility. He looks back at the explanation window on 
the ES display to try to understand the data used and the reasoning 
followed in arriving at its diagnosis. The explanation scrolls beyond the 
window. He begins to read it, but then turns to the process display to check 
on how things are progressing, and to determine what the next 


4 Note that the male pronoun is used only for simplicity's sake. 
Actions to place the system in a "safe" configuration. 
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management action should be. As he looks at the raw data he realizes a 
new event occurred. Was this due to an action taken by the automated 
system. He turns back to the expert system to check the status messages; 
there is a long list of messages. He searches and scrolls among the 
messages to find one that might explain the current pattern. Meanwhile 
data on the process display are changing... 

"Clumsy" Explanation 

This example attempts to give a sense of the flow of cognitive activities 
involved m managing disturbances. 6 A model of the cognitive activities in 
disturbance management is shown in Figure 1 (from Woods et al 1991) The 
first step is for the practitioner to recognize, out of all the potentially changing 

in^!w°H nS ' W ^ 1Ch af f an °P ialles - Recognizing one or more anomalies leads § 
cognitive activity about how to cope with the disturbance (s), i.e., what 

sa ing responses to execute. When these actions are taken, the practitioner 
needs to monitor to see if the responses have occurred as expected and 
whether they are having the desired effect. If unexpected anomalies are 
diagnostic cogmtive activity is triggered. When a diagnosis is reached, the 
lines of reasoning concern developing corrective responses. Note how the 
practitioner m the example needs to cope with the consequences of faults and 
perform fault diagnosis in parallel (Woods, 1988; 1994). 

The example illustrates a lack of coordination for joint problem 
solving mat can exist between a human practitioner and an "intelligent 
system. The first problem has to do with the time period during which the 
expert system's diagnosis and explanation arrives. A retrospective 
explanation typically comes at a time when the practitioner is likely to be 
engaged in multiple activities which may include several of the following- 
evaluating hypotheses, dealing with a new event or the consequences of the 
rault(s), planning corrective actions or monitoring for the effects of 
interventions, and attempting to differentiate the influences due to faults and 
those due to corrective actions. The expert system does not coordinate its 
attention direction behavior with the activities of others by judging 
interruptibility the way people do (Pavard et al., 1989). In effect, the system's 
message occurs as a potential interruption to these on-going lines of 
reasoning and monitoring. 


particular o^raHng^^jtext! C ° nditi ° nS 411 which P rocess state deviates from the desired function for the 
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Data Channels 





Figure 1 . Model of anomaly-driven information 
management. 


processing in dynamic fault 


j* A s . eCon , d P rob,em concerns the amount of data present in the 

A l0n 8 explanation requires theprachtionerto pay 
.f" ° the ex P ert system for some time, demanding cognitive resources 

to follow the reasoning. In addition, interface management tlsks mav be 
required such as scrolling, in order to read the complete messa« or such as 

undLTaIdZTSm 0 " 5 i""”" in ° rder t0 

of the from prS," 0 " 

because it means that establishing relationships among the Zeis daTa and 
Z P a i at ‘° n needs t0 be “Clertaken as extra tasks by thi practitioner X, 
nrZtT 11131 exa ™ mn 8 1,16 diagnosis and explanatio/will take the 

StiniTn Z7ed r ° m W L ha ,' iS CUrren ‘ ly g0ing 0n in P r °cess, possibly 
sZw 8 ^ d eVe ” In 8 ene ral, the dynamics and form of this expert 

basil S t x ? anahon make lf a "clumsy" type form of interaction. This term is 
based on clumsy automation," a phrase coined by Earl Wiener to refer to 
automation that provides its benefits during low Workload periods, but 
a es new urdens during high workload periods (Wiener, 1989). A clumsy 





iS 0n f e i hat CTeateS extra workioad during high tempo 

Le and ,££ a^dt^r tT ^ " • busy 

concept. ea es extra tas s and cognitive burdens. Figure 2 illustrates this 



Figure 2. Clumsy Explanation." 


Feedback 

about whlfer'lS lhe P raCtltioner w» unsure 

Effective feedback on SS£“ “ SS,“ ^TT , 

may f “ 

actions taken by mher "enS ^ dUe ‘° 

hi been a 
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Some Characteristics of Typical Expert System Explanations and New 
Directions 

Recent AI research has pointed to some of the problems associated with 
typical expert system explanations. Many of these criticisms and subsequent 
developments in the research labs have been inspired by studies of human 
explanations (e.g., Pollack et al., 1982; Cawsey, 1992; Paris, 1988). Much of the 
work in explanation concerns developing better explanations for diagnostic 
systems (e.g.. Hasling, Clancey and Rennels, 1984; Chandrasekaran, 1989) or 
better explanations for how some device works for tutoring purposes (e.e v 
Cawsey, 1992, Feiner and McKeown, 1990; Suthers et al., 1992). These task ' 
domains have important characteristics that distinguish them from process 
control: 1) the underlying system is static and unchanging, and 2) time is not a 
significant factor. It is worth noting that there seems to be no research that 
investigates what constitutes effective explanations in dynamic fault 
management applications. We can expect the characteristics of the task 
demands to impact cognitive functioning and hence to impact the nature of a 
cognitive support system. In particular, in this work we are concerned with 
how the demands of process control impact the nature of effective AI 
diagnostic support. 

Below we summarize some of the characteristics of typical explanations 
that are problematic for dynamic fault management. We also point to AI 
research that has also indicated the limitations of these characteristics for 
other domains. 

* Explanation as Retrospective. Expert system explanations are typically 
given at the end of some problem solving activity (Cawsey, Galliers, Reece 
and Jones, 1992). These authors point out that, by contrast, when people are 
en g a ged in collaborative problem solving, they tend to provide justifications 
of their beliefs or reasoning as part of the problem solving. 

In dynamic fault management, practitioners often cannot wait for a 
full-fledged diagnosis before taking corrective actions; though the picture of 
the underlying fault(s) may become clearer with time, the ability to recover or 
safe the system will tend to decrease with time (Perrow, 1984). Hence, some 
situations demand that management actions be taken before all the data 
becomes available or with uncertain data. Some understanding of the state 
and evolution of the process is needed in order to move the situation towards 
stabilization. This suggests that there should be collaboration in the diagnosis 
process, that is, with hypothesis revision and refinement over time (Malin, et 


The plane went into an uncontrolled roll and lost thousands of feet 
aircraft. 


in altitude before the crew recovered the 
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process of the AI system'may 1 resuhina ^ , P,° P on the Interpretation 

taking the practitioner awaWrnm th 'gher workload because it will mean 

time in order to read a long explanation^ evaZteits' sotiXT^ 

related from i Frpiainer to Evp l,; A 

shot" or as one o^ chunk fZ nV* lhe >' are Prided m "^e 

Mastaglio a“ veS S Bv To 7 f Wart ° Ut ' 1989; Caw *y< 1991 - 

provide explanations show thaUt often requhS? dhd^e° f how P eo P le 
example, clarifications or elaborating (\a ^ , dialogue consisting of, for 

1991). In a sense then! an exnlanahon is , “ dSwartou ‘- 1990; Cawsey, 
explain ee is involved in the shaDine of the™ 80 ! 113 ' 1011 pr , 0Cess ' m which the 
Recent research directions include how ?* ex P lanatlon (Pollack et al., 1982). 
interactive (e.g., Cawsev i Mo„l h ’ make machine explanation 
Cornell, 1992). ' ' and Swartou t, 199 °; Suthers, Woolf and 

too mn^ irUo^^e^lanati^n'^ey^^T^par^e^wi^^his^tTO'^f aC ^ ^ UtS 

By contrast people provide "minimal" g ,. ^ an t to know about a command, 
jtrst ^information needed (Mastagho 

forces the uTeTtowlmh^oTand 0 eXplanalion resul ‘ s in data overload; it 
no guarantee that the i^rmahortheus'e* 7““ information ' “d there is 
particularly problematic “ 

present, and in which the consequences^ erroneo^ctionTcan b"^" 

imphdU^^^response^some P que^^g^ 0 ^how n ^«wh U8 ^r') > ^TV l^ aSt 

(Malin et al., 1991 V ol 2) Bv rnn ? * xplicitly requested by the operator 

"spont^eous"^xpknadon/ (Kar^enty^r^Falzon 1 1992 )^ ^Pf^Pl d or 
providing explanations only when queried I i'n A P roblem Wlt h 

what question to ask (what^ information they stk)^r ta'tW T ^ ^ 
may not even know that thev shm,M h* , r the extrem e case, 

need some information). This is particular! ^ ex P lanation (that they 
situations in which several aeenkmav Hp in e , eVant ln com P lex dynamic 
and in which situations can evolve quickly. V ° ^ “ mana 8 in g the P^cess 

typical *exp ^ations ^ ven^v tlTV 0 ^ A premise behind 
on what is being explained and P provWesTe pe“ tfa fLl 
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perspective is inappropriate for supporting cooperative fault management in 
which practitioners are highly knowledgeable about the monitored process 
and are active in its management. Rather than substituting for expertise, 
many researchers believe that a more appropriate interaction paradigm is to 
support practitioner expertise (Roth, Bennett and Woods, 1987; Billings, 1991). 
Insight may be gained from studying collaborating team members with 
different levels of experience and expertise, and who have overlapping 
expertise. 

• Explanation as Context-Independent. Traditionally, the goodness of 
an explanation has been seen as context-independent, i.e., that there is a valid, 
best explanation for some event (Leake, 1991). Leake points out that in this 
approach, determining an explanation relies on certain criteria such as 
explanatory coherence or testability, completeness, and brevity. Work on 
explanation in philosophy and psychology (Hilton, 1990) and more recently in 
AI (Leake, 1991, Cawsey, 1991; Moore and Swartout, 1990; Suthers, Woolf and 
Cornell, 1992) indicates that effective explanations need to take into account 
the goals and information needs of the explainee. 

Another sense in which typical AI explanations are context- 
independent is that they are dissociated from the process views. For example, 
it is common to present IS interpretations and explanations in a separate 
"message list" window, i.e., a list of text messages (Malin et al., 1991). Message 
lists often fail to highlight monitored process events and relationships (e.g. 
temporal, causal) among these events (Potter and Woods, 1991). This lack of 
coordination among the IS assessments and the process view imposes the 
integration task on the practitioners. In such cases, practitioners simply have 
ignored the intelligent system (Remington and Shafto, 1990). 

Diagnostic support systems for dynamic fault management that 
provide explanations with the above characteristics for dynamic fault 
management are likely to lead to "clumsy" interactions (Malin et al., 1991; 
Remington and Shafto, 1990). Woods (1989) points out that disturbance 
analysis systems developed in the nuclear industry in the 1980's, failed for the 
same basic reason. Though these systems did not incorporate AI, they 
attempted to automate diagnoses. The actual result was clumsy explanation 
and data overload-- operators had to sort through the diagnostic information, 
interpret it, and integrate it with other sources, all at the same time that the 
demands for monitoring, assessing and responding to events in the 
monitored process were the highest. Similarly other studies have found that 
technology purported to aid practitioners in fact created new cognitive 
burdens during critical and high tempo portions of the task (Woods, 1993a). 
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Overview of Approach and Scope 

experienced practitioners, onetan fo m C^" 8 

the perfonnance ^^cti^ners^ngaged^ac^f 

interventions. The study is whatmilhtbe called lakm 8 any 

one particular aspect- how practitioners ,1^! deScrlbed hereln focuses on 
anothers' situati 0 P n assessment ™ 

Contributions 

S££ 55? ^ : =5S5 r ‘ 

S2^^g==s3ST 

^ssaaesga 4 »-^“S3zr-^‘' 
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Figure 3. Basic Logic of the Research. 
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Overview of the Rest of the Report 

Chapter 2 presents the concept base (or conceptual framework) used to 
analyze the raw data. This framework consists of concepts that highlight the 
cooperative nature of communication, i.e., the context sensitivity of 
explanations and the notion of a "common ground." Chapter 3 presents the 
research strategy of the study. Chapter 4 presents the findings and Chapter 5 
discusses these findings. 
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CHAPTER II 

FRAMEWORK FOR ANALYSIS 

“fn^^ 

and which can generalize to other (Lmah^V^a* 0 3nd domain-independent 
have a framework or %&!&”" - ds * 

importance (Hoilnagel et al„ 1981; Wood! S 1993b) ® concepts of 

explanahorfcapa^L^^aifbe^tfght^of^e^generan^as^cormn 311 ^ 

problems. Therefore thp mnronhni c 7 o enera *v/ as communication 

on research and «“* d -s 

conr^rsadon and'explan^hon as a^ 6 ma, '° r “"“P' 8 °' ,he fr3meW0rk •* 

ground" tC durinv TP" 3 * 1 ™ endeavor and the "common 

explicit some concepts that seem to be Teteramto mdmtlndTnT 0 ^ makeS 
communication and cooperation in distributed problem solving 

Cooperative Communication 

utterar^^ ( ^^^ C ^ d ^^^^ dd ^^®^ d ®”^^^®^hehhnd° rC * er ^ 
referred to' three !rmaM^° o a f? here ‘° P ™ ci P les or maxims. Grice 

T.ZZ: Speak as Clearl >' as Possible, avoid ambiguity. 

• ~ but not more 

‘speaSg make y0Ur COntributio " relevant to the context in which you are 

accoun'^^omatioif 1 ^ur a ^) S ii^^, k f5 the& T* * ^ 

tailor their communications to what they perceiveTb! th® m f 5ages ' Feo P le 
needs of their listeners (e.g„ Krauss and Fureell, TOl“ lnfor mat.on 

allow oa!to S aS 1° human - c omputer communication, 

how they a° o m P °v r p mteract '0»- Woods (1992a) illustrates 
(e.g„ "Error code £££■) SS? ^m^ / ,nscrutable alarm messages 
clearly what the proWem " r '!,n , ° f the y do n ° l «Press 

several different underlyme problem “i “ ^ the Same ala ™ indicates 
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communication; for example, in the case of the group alarm, the person will 
need to seek out the missing information to understand what event in fact 
has actually occurred perhaps during a high tempo period. 

The Cooperative Nature of Causal Explanations 

Any particular event may be "explained" in different and valid ways. 
There are potentially many contributing factors-- each necessary but only 
jointly sufficient to produce the event that is to be explained. We often speak 
as if there was a single cause for the event, but we are selecting one of the 
multiple necessary factors to focus on. Which of these multiple contributors 
we select depends on the purposes of our inquiry. 

Causal explanations always have an assumed contrast case, a "rather 
than" built into them (Hilton, 1990). For example, an explanation to the 
question "why did x occur?" contains some counterfactual contrast case, even 
though it may remain unarticulated. The complete question, if articulated, 
could be one of several, such as: "why x rather than not x?" , "why x rather 
than the default value for x?" or "why x rather than y?" The factors that are 
seen as mere conditions and those that are viewed as causal will vary 
depending on the frame of reference adopted, i.e., on the contrast case. Rather 
than explaining an event per se, one explains why the event occurs in the 
target case and not in some counterfactual contrast case. What is taken to be 
the cause depends on the causal counterfactual contrast case (Mackie, 1965). 

According to Hilton (1990) causal explanations are like other forms of 
dialogue in the sense that they adhere to the general rules of (cooperative) 
conversation, or Gricean maxims (Hilton, 1990). To be relevant, explanations 
must close a gap in the explainee's knowledge concerning the issue in 
question (Turnbull, 1986). Some conditions are background (mere conditions) 
and some are foreground (causes) relative to the purposes of the explanation, 
which generally depends on the information needs of the explainee. The 
relevant explanation is one that "refers to the factor that makes the difference 
from the questioner's point of view between the target event and the 
backgrounded contrast case" (Hilton, 1990). Another way to say it is that some 
conditions are taken as shared or common knowledge, whereas the 
explanation itself focuses on the factor(s) that the explainer believes the 
explainee needs to know. Therefore, the goodness of an explanation depends 
on whether it provides the information that is needed to satisfy some goal 
(Leake, 1991). 

Studies indicate that people provide explanations that attempt to close 
the gap in someone's knowledge and furthermore that they will change the 
explanations they give depending on the knowledge that they perceive to be 
shared among themselves and the explainee (see Turnball and Slugoski, 1988 
for an overview). So if explainer and explainee share knowledge of some 
contributing factor x, then factor y which was also involved but which the 
explainer thinks the explainee does not know, will be emphasized. These 
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the communication efficiencies afforded by the increased use of [deitic 
reference, i.e., pointing]; the visibility of a partner's task relevant action 
constraining the range of meanings attributable to an utterance." (p. 212). 

This work is consistent with findings about the importance of open (or 
observable) interactions in team performance (Hutchins, 1990). The particular 
interactions afforded by the task environment and tools affect the nature of 
grounding. In the domain of ATC operations, Hughes et al. (1992) point out 
that the tools used (e.g., flight strips) allow for open interactions-- they allow 
all the relevant participants to easily see the state of the system and to see 
what actions others take on the system. On the other hand, some 
characteristics of work environments may inhibit the ability to ground. For 
example. Woods et al. (1994) point out that multifunction controls and 
displays used in the cockpit tend to suppress cues about the activities and 
intent of the other human crew member. This disrupts their ability to 
maintain a common situation assessment, which can degrade 
communication and coordination across the crew. 

Grounding in dynamic fault management 

In dynamic fault management there is an external reference: the 
process being managed. Not only must team members maintain an up-to- 
date common ground about the state of problem solving, they must also 
maintain common ground about the changing state of the monitored process. 
Many monitored processes cannot be observed directly, but only via data 
derived from sensors, transducers, etc. 

Another difference is that in dynamic fault management applications 
there are typically several data sources which provide different levels of 
processed data; raw data may be available from computer-based displays 
(monitored process displays), and processed or integrated data (in the form of 
assessments) may be provided by "intelligent systems." Maintaining common 
ground about the state of problem solving and about the state of the 
monitored process requires knowing about the relevant activities of other 
team members because their activities may impact the process. This is 
important in order to be able to manage the process effectively (because 
knowing what to do depends in part on knowing what has been done, what is 
expected and what is planned for). It is also clearly important for diagnosis (in 
order to know what may be the causes(s) of an anomaly). Furthermore, team 
members also need to be grounded about relevant assessments of others 
because these can potentially affect expectations and plans. 
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CHAPTER III 
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anesthetized for the duration of the operation and to maintain normal 
physiological function (e.g., intravenous fluid to replace blood loss). During 
emergence, when the surgical procedure is finished, the administration of 
drugs is discontinued and the patient is awakened and is extubated. 

Anesthesiologists are the physicians responsible for managing the 
p ysiological process and diagnosing unexpected anomalies during the 
operation To do this they need to monitor the patient’s physiological signs 
and the effects of anesthesia and surgery. The major functions and signs that 
anesthesiologists must monitor are: depth of anesthesia, circulatory function 
blood loss, respiratory function, respiratory and anesthetic gases, renal 
unction, neuromuscular function, body temperature and other system 
functions depending on the type of surgery or the health of the patient (e g 
blood sugar, electrolytes, hemoglobin.) Clinical means (e.g., inspection 
palpation, auscultation) as well as several instruments are used to provide 
indications of these functions and signs. Some important devices and 
monitors used include: 

• electrocardiogram (ECG) to monitor cardiac rhythm, 

• pulse oximeter to measure blood oxygen saturation and pulse rate 

• arterial cannula or "A-line", which measures arterial blood pressure, and is 
used to sample arterial blood which is sent to a lab for analysis of partial 
pressure of oxygen and carbon dioxide in the blood, 

• pressure^ Sphygmomanometer (i e ' inflatable cuff for measuring blood 

• central venous pressure (CVP) catheter, which provides an indication, 
together with blood pressure, pulse rate and urine output, of blood volume 
and hence a guide to fluid replacement, 

• capnograph to measure airway concentration of carbon dioxide, 

• mass spectrometer to measure and distinguish the concentration of various 
other gases. 


• thermistors to monitor body temperature, 

• Swan-Ganz catheter for measuring pulmonary artery and capillary wedge 

pressure and cardiac output. 7 8 

Data from many of these measurements are available on an integrated 
computerized display (which we refer to in the text as the vital signs 
monitor). While some of these data are continuously available (e.g., the heart 
rate) others are available at intervals (cuff blood pressure) and still others 
require some explicit activity by the practitioner (e.g., cardiac output). Also 
not all data is immediately available. For example, an arterial blood gas 
sample requires analysis in a remote lab and ten minutes may elapse between 
drawing the sample and receiving the results. 

v M T Sement actionS/ suc h as administering drugs, blood, or fluids, are 
taken on the process depending on its state. Many drugs, each with specific 
actions, side effects and contraindications, are available to the 
anesthesiologist. Some types of drugs that are typically used during the 
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hypotensive agents, vasopressors and vasodilators. ' 

tat , F ° r .T/® 0n the cognitive activities of anesthesiologists see Cook 
Woods and McDonald (1991) and Xiao (1994). 8 C k ' 

Practitioner Roles and Relationships 
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Supervisory Control Issues 
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duringthe whole operation, while the attending is present only some of thp 
t me (since he supervises other cases as well). When he return^ to the 

P^nfT' he needs j° u P date his situation assessment (e.g., determine what 
will typi 3S:t^ ^ Pr ° Ceedin ^ the “ 

subordma es take control actions on the process, and may provide 
assessments to the supervisor. Figure 5 shows the similarities among the 
resident-attending relationship and the supervisory control relationfhk, In 
the operations involving a senior and a junior resident, the relaSwn 
among them was similar to that of an attending and resident in the senfe that 
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Figure 5. Supervisory control relationship among attending and resident. 
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• receives and implements instructions, 

• provides feedback on actions, 

• offers assessments, 

• updates the supervisor when his attention returns to the process. 

Types of Joint Cognitive Activity and Information Exchanges 

Anesthesiologists continually monitor and manage the patient's 
physiological process and diagnose unexpected anomalies. In order to 
perform the high level goals of management and diagnosis effectively, 
practitioners in this domain must form expectations about the future plan 
courses of action, keep track of what has occurred, and evaluate past 
management actions and interventions. These cognitive activities are 
re ected in the information exchanges seen among team members. Figure 6 
illustrates these types of joint cognitive activity (and information exchanges) 

f,™ ^ he * r role in servin g the goals of management and diagnosis. The arrows 
hat feed into management and diagnosis indicate cognitive activities that 
support management and diagnosis. The figure depicts past, current and 
uture views into the process, i.e., patient's physiological state as monitored 
via vital signs. Updating, in which one team member informs another about 
relevant events that occurred during his absence, concerns past events 
t ough it is driven by the goals of the present. Team members also talk about 
the past in another important way-in order to evaluate the effects of their 
interventions. 

Practitioners establish expectations about the future of the process and 
about what actions should be taken; these expectations and plans 2 are 
incorporated into the present actions (Cook, Woods, and McDonald, 1991; 
Xiao, 1994). Information exchanges about the future reflect the need to 
anticipate problems that may arise, and to prepare courses of action to avoid 
them or be better equipped to deal with them (i.e., contingency plans). Some 
of these problems may be unlikely, but practitioners prepare for them either 
ecause the consequences are high or because coping with these problems 
when they tend to arise would be too resource-consuming. 


n ° ted that u P lans for courses of actions can be formulated . 
to the contingencies of the current situation (Suehman, 1987 ). 


varying levels of detail and subject 
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Figure 6. Different Types of Joint Activity and their Relationships. 

i T V t0n i ! 1 fractions are another type of information exchange 
observed in the data. These are exchanges about domain knowledge 8 given for 
the purpose of instructing a less expert practitioner (or would-be-pmcSer) 
y are not critical to the moment-to-moment management and diagnosis. 

The Field Study 

Ten neurosurgery operations were observed and videotaped The 
neurosurgeries involved one of the following: clipping of a cerebral 
aneurysm , removal of a brain tumor, or a laminectomy.* A list of these 
operations, along with some particulars, is given in the Appendix The 

5 atten^nl?!^ 1010 ^ staff Reived in these operations was comprised of 
ttendings and 5 residents. The practitioners were recruited by the second 
author who is also an anaesthesiologist working within the same 
organization. The data collection was part of a larger study of physician 
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interaction with computers and physician expertise. Note that they were not 
told that this was a study of communication or explanation. All but one 
resident agreed to participate in the study. 

A transcription of each was prepared from the videotapes. The verbal 
behavior was integrated with (a) other data about patient state, (b) physician 
activities, and (c) goals and intentions of the practitioners based on domain 
knowledge, procedures, typical practices to form a behavioral protocol 
(Woods, 1993b). 6 

Note that this study does not take a hypothesis testing approach. 
Rather, the approach involves posing guiding questions which define 
episodes of interest, which in turn, are analyzed given the conceptual 
framework. 

Guiding Questions 

The impetus for this study was the problem with the conventional 
approach to AI diagnostic assistance, in which diagnoses or recommendations 
are provided along with retrospective explanations for dynamic fault 
management. Hence, the questions addressed in this investigation are: how 
do human team members provide diagnostic and management support in 
dynamic situations? What is the nature and function of their explanations? 
We can further refine the latter question, by distinguishing between 
interpretations and explanations: How do team members communicate about 
interpretations and assessments? How do team members provide 
explanations for their assessments? How are these two kinds of explanations 
used in joint problem solving or in establishing situation assessment? 

An important aspect of situation awareness in distributed dynamic 
fault management is keeping aware of the relevant actions that others have 
taken or will take because these may impact the process or the state of 
problem solving. Hence another guiding question is: How do team members 
keep informed about the relevant actions of others? 

Data and Analysis 

Data Sources 

The data sources relied on in the analysis are: 1) verbal 
communications made by the anesthesiologists and those directed to them by 
others, or those that may have been overheard by them, 2) actions taken by 
the anesthesiologists, such as: looking at the monitor, any interactions with 
the machines, any adjustments to drugs or objects, any samples taken (and 
how taken), or drugs given, 3) actions taken by other personnel when 
interacting with the anesthesiologists, 4) behavior of the dynamic process as 


researchteam ' l ° ^ anaIys,S that an anest hesiologist who also worked in the organization was part of the 



26 


indicated by the patient record kept during the operation, and as displayed by 
the various monitors and machines, and a record of vital signs. 

Three cameras were placed in the operating room so as to capture these 
data sources. One camera was focused on the various anesthesia machines 
and displays. Another focused on the area at the head of the operating table 
and in front of the machines, where the anesthesiologists spend most of their 
time. Finally, another camera focused on the patient, which captured close-up 
actions taken on the patient. 

Transcription 

The guiding questions drive the episodes selected for analysis, as well 
as the transcription process to some extent. It would be misleading to speak 
about data acquisition and data analysis as completely separate; during the 
transcribing, some data analysis occurs in the sense that the data is transcribed 
with certain omissions, while capturing and detailing certain behaviors. As 
Jordan and Henderson (1994) point out, there is no ideal transcript-- " .it is 
impossible to include all potentially relevant aspects of an interaction, so that, 
in practice, the transcript emerges as an iteratively modified document that 
increasingly reflects the categories the analyst has found relevant to her or his 
analysis." 

The videotapes were transcribed in two passes. The first pass consisted 
of transcribing all verbalizations made by the anesthesiologists and 
verbalizations made by other team members to the anesthesiologists. The 
only verbalizations omitted were those that were obviously chatting about 
social activities or gossip. These were summarized as such in the transcript. 
Note that although we focused on verbal behavior in this study, the analysis 
is not a linguistic one (i.e., how language is used). Rather, it concerns how 
information exchanges function to support dynamic fault management. Also 
recorded were various activities taken by the anesthesiologists, including 
interactions with the machines, or other equipment, or administration of 
drugs or other fluids. 

The next stage involved reviewing the transcript to identify particular 
episodes (which are explained below.) Having fairly detailed transcripts in the 
initial pass is useful because once an interesting episode is identified, it allows 
one to look back within the transcript to see what other episodes, activity or 
events may be related. Then these episodes were reviewed on video to verify 
the transcription and to fill in more detail about activities and vital signs. 

The goal of field studies in cognitive engineering is to be able to 
generalize findings to similar situations in different domains (Hollnagel et 
ah, 1981; Woods, 1993b). This involves taking a description of the actual 
performance (episode in raw transcript), which is concept-free and highly 
context-dependent and converting it to a description that is more concept- 
specific and domain independent. In order to arrive at a concept-specific and 
domain-independent description, one needs to have a framework or 



27 


presentedHn ^Chapter^Two" “""P*' ™ S framework "as 

analysis. The ° f lhe l0gic of * e 

contained within the text Thic Ho ^ e P 1S0( * es °f interest is 

episode in do“ d^dem * TaTinf* * 3 re P res ™‘ ati ™ of the 
terms. penaent terms and, in some cases, in problem solving 


• Guiding questions 

• Theoretical framework 


t 

Transcript 





t 

Episodes 



J 

*• Patterns & higher level 
concepts 



Figure 7. Basic Logic of the Analysis. 
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the data (e.g., blood pressure is 90) but a form of interpretation because being 
low depends on the context- low for our goals and expectations given this 
patient in this circumstance. 7 Explanations are another type of inform; these 
refer to explanations of interpretations, i.e., explanations for why some 
interpretation is held. 8 

Updates 

Updating episodes ("updates") are situations in which a returning team 
member (typically the attending) is updated about the state of the process 
and/ or of problem solving. Updates are selected for study because they are 
found in many fields of practice in which a process must be turned over to 
another crew while on-line (e.g., air traffic control, nuclear power plant 
control room operations.) 

Updates are relevant in supervisory control situations in which a 
supervisor periodically monitors several intelligent subordinates. The 
technological trend is for fewer people to monitor interacting subprocesses 
through the use of increased automation and intelligent subordinates. This 
means that supervisors may be coming into "advanced" situations. In some 
cases, the supervisor will have to be called into a situation which has 
escalated beyond the ability of the intelligent subordinate to cope with it. 
Effective updating is a desirable capability for such an intelligent subordinate. 

Updates occur when the attending comes back into the operating room 
after having been away for some time. 9 Recall that the attending is present at 
the beginning of a case (during induction and intubation) but then may leave 
to oversee other cases. When the attending returns, the resident may update 
him on the progress of the case, and/or state of the patient. Particularly 
interesting are those interactions/ updates after some critical event has 
occurred. 

Management and Diagnosis 

Monitoring and management occur continually. Of particular interest 
are episodes in which two or more team members are engaged in managing 
the process and/ or in anomaly detection. The beginning and end points for 
an episode are not well defined a priori. However, generally speaking, this 
kind of episode will begin with a focus of attention on some anomaly, and 
end when an interpretation is arrived at and/or management action is taken 
and the topic is dropped, resolved or otherwise attains some closure. 


n 

Note, however, that one cannot say that an utterance such as "blood pressure is 90" is always a statement 
of data. In some contexts such a statement can be intended and can communicate to someone else an 
interpretation or event (i.e., that it is low). 

^These explanations may not occur with the telltale words typically associated with an explanation (i e 
because , "the reason is..." etc.) r ' 

^There is another kind of update found in the data; this occurs when a team member comes to relieve the 
resident. This will typically be another resident or a CRNA (certified registered nurse anesthetist). 
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utterance may take on any of several meanings depending on its context, the 
meaning or significance of many episodes cannot be understood without 
knowing their context (e.g., what occurred previously in the case, what 
parameters have been of concern, what practitioner expectations are.) 

The main type of representation used in the Findings section provides 
a segment of the transcript (corresponding to an episode) with footnotes to 
assist the reader in understanding some of the domain-dependent details. 
Also, to assist in the analysis, a domain-independent description is provided 
alongside the transcript. Episodes that involve diagnosis also contain a third 
column, indicating phase of problem solving. Some episodes are short and do 
not contain additional columns beyond the transcript. The conceptual level 
description of each episode, i.e., why the episode is significant for the 
purposes of the study, is contained within the text. 

The utterances in the presented transcripts are not timestamped 
(though this data is available from the videotapes) because the dialogues in 
the episodes typically do not have long pauses between utterances. Where 
relatively longer pauses are found, these are noted. 

An identifying code is used before each episode. The code indicates the 
case the episode occurs in and the transcript time, as follows: 

[case I hour: minutes: seconds]. 

Key to Transcription Symbols 

There are several different transcription schemes that could be used, 
each capturing different aspects of speech and dialogue. For my purposes, 
however, a rather simple scheme given below suffices. Aspects of prosody 
were captured in the second pass transcription, if they were deemed relevant 
to the analysis. 

• Ellipsis indicates missing, inaudible or incomprehensible text. 

• Ellipsis in parenthesis indicates approximate number of 
incomprehensible words represented by the dots 

• Italics indicates actions 

• Words in parenthesis express some uncertainty about the actual words. 

• R = resident (used in cases where there is only one resident) 

• RS = senior resident, RJ = junior resident 

• A = Attending 

• M = Medical student 

• S = Surgeon, SA= assistant surgeon 

• N = Nurse 

• v.s. monitor = vital signs monitor, an integrated monitoring system 
that displays all the patient's vital signs. 
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CHAPTER IV 

FINDINGS OF THE FIELD STUDY 

The introduction pointed out the problems both with the form and 
iming of explanations in the standard expert system approach to diagnostic 
assistance. This problem defines the guiding question for the investigation: 

w T ° t6am members collaborate in management and diagnosis? 

We begin by examining episodes in which team members are eneajdta 
fornung interpretations of the process state. Recall that these are one type of 
explanation relevant in dynamic fault management. Then we examineVhe 
characteristics of explanations for interpretations or actions, wSre the 
counterparts to AI justifications. These findings lead to an investigation of 

important factors that play a role in the nature of diagnostic support among 
team members m this domain. ** ® 

Explanations in Dynamic Fault Management 

Ioint Interpretatio ns about the Process Slate 

thaf mfihf K de ! ° f management and diagnosis reveal instances of interactions 
that might be termed joint interpretation. This term attempts to capture three 

of interaction: it is a process in which team members 
are involved in coming to the mutual acceptance of an interpretation. 

Episode: Anomalous blood pressure 

in iaVp C a ° nSider the epiS ° de in RgUre 8 ‘ In ** e P isode ' determining whether 
to take a management action and what it should be, depends on the exact 

nature of the interpretation for the anomaly. This interpretation Ts not sUlv 
transferred from one agent to another. Rather, both are engaged in the process 
of a rnvin g at the interpretation and both contribute to its development 
ough the senior resident proposes an initial interpretation ("they must've 
stimulated something"), it is followed by a period of investigation and 
verification in which both team members are involved. Thf junior resident 

Hnp! Pt 1 ( nV ° V ? d by th j senior resident ' in the sense that the senior resident 
d 0t ^P ly P rovide directives, but also states his assessments and 
easons. The junior resident's comments and actions provide evidence of 
attending and comprehending. For example, she implies understanding 
when she points out corroborating evidence. 

The subsequent exchanges concern further verification and testing that 
relate to establishing that the cuff pressure measurement, rather than the 

nrnv-H^ a- the artlfactual one - This phase ends with the senior resident 
v V mg 3 directive for corrective action, along with the statement "I think 

direct^ P “ WhiCh Can be COnsidered an explanation or reason for the 
mutuallv ^ thls con t ex t it functions more as a confirmation of the 
, 7 e d interpretation, because understanding the statement depends 

on the information that has been gained through the joint interpretation. 
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TRANSCRIPT 

[ W\ 4:22:38 ] 

{R2 looks at v.s . monj 
R2: [R] 

R: yes? 

R2: his pressure's now 
reading 177 

IR2 hits b.p. button to start 
cuff measurement) 

R: They must've just 
stimulated something. 

(R2 adjusts anesthetic agent , 
gets syringe) 

R: Don't give him anything 
yet, see what the cuff 
pressure is 1 . It's a lot better 
waveform than we were 
having, so I think it's 
probably true, they 
stimulated something (both 
looking at v.s . monitor) 

R2: yeah, his heart rate 
picked up 5 points (indicates 
to monitor) 

R: yeah, you’ll see that 
when they’re doing cervical, 
especially anterior, posterior 
not so much, but the anterior, 
you’ll definitely see, you 
gotta be looking for vagal 
stimulation, you got the 
vagus. ..you got the carotid 
(body), and you gotta be 
watching for all those 
things, it's just a real touchy 
surgery..this is not abnormal 
at all 

[continued] 


DOMAIN INDEPENDENT 
DESCRIPTION 


R2 draws attention to anomaly 


R suggests interpretation 


R tells R2 to wait to take 
management action she was 
about to take, saying they 
should check another 
parameter (because 
anomalous data may be 
artifactual). But also 
comments on qualitative 
characteristic of data 
suggesting accuracy of 
reading 

R2 calls attention to a related 
parameter that suggests 
anomaly is not artifactual 


R mentions conditions under 
which flucuations in 
parameter would be 
expected 


* Blood pressure is measured by two sensors; from an arm cuff and from 
displays blood pressure continuously as a waveform. The cuff pressure, 
measured intermittently. When an arterial line is present, cuff pressures ; 
minutes. 


PHASE 


anomaly 

identifed 


hypothesis 

suggested 


verification 

/testing 

hypothesis 


corroborating 
evidence 
pointed out 


i the arterial line. The arterial line 
in contrast, is a discrete value 
are measured typically every 15 to 30 


Figure 8. Episode: Anomalous blood pressure. 



Figure 8 (continued). 


TRANSCRIPT 


DOMAIN INDEPENDENT 


PHASE 



R2: his blood pressure cuff is 
making me crazy 

R: they correlated well until 
we started surgery and now 
neither one of them 


R2 comments on second 
sensor value 


R2: yeah 


R: ..(probably cause his 
arterial) pressure's changing 
so rapidly, plus they're 
leaning on his arms.... 


R calls attention to data 
indicating why second 
sensor value is artifactual 


uerification/ 

testing 


R 2.: (R2 looks back) right 
there 


R2 corroborates latter 
hypothesis 


R: cuff can't read his pressure 
well. Give him another 50 
mics of fentanyl. 1 I think it's 
a true pressure 
{R2 administers) 


R directs R2 to take a specific 
management action 
consistent with the 
interpretation, provides 
explanation 


correctiue 

action 

directed 

and 

taken 


R: (hits b.p. button) gonna 
drop his cuff, it's just 
complaining too much. 


R cancels the second sensor 
measurement (which has 
been deemed artifactual) 


1 A narcotic which blunts the response to stimulation. 


Epode: Evaluating the effects of i ntervention. 

The senior resident relates these results to previous imelvenTons in an 

b^o^ 

( turning down the nitro was a good thing ") 

Notice™\eMomh!n 0 h i el Strate .l an ilUerplay betWeen data and valuation, 
e relationship between the junior resident's calling attention to 
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particular data values and the senior resident's cognitive activity in response: 
evaluating possible management actions, developing further management 
actions, or verifying states (e.g., when the junior resident queries the senior 
resident about the low temperature, the senior resident engages in checking 
behavior -to see that the blood warmer is on). So, far from distracting the 
senior resident, the junior resident draws attention to data that assists in 
forming an interpretation of the process. Through the senior resident's 
voicing of his assessments, both team members can become calibrated about 
t e state of the process and management, or, in the language of the 
conceptual framework, they add to the common ground. 

Epis ode: The state of management 

The next episode (see Figure 10) illustrates how team members keep 
one another aware of the state of management activities. The episode begins 
with the resident informing the attending of an intended action. Team 
members do this relatively often. Depending on context, this serves several 
possible functions: it allows for confirmation that the action is an appropriate 
action at the current time or alternatively, it allows the other team member to 
halt or modify the proposed action. It adds to the common ground- the other 
team member knows what will be done and hence can modify expectations 
appropriately. The joint discussion of management strategy in this case makes 

explicit particular concerns and engages "two heads" in thinking about what 
to do. ° 

The information exchanges among team members always occur within 
a dynamic physical context. Even while talking to one another, practitioners 
can continue to monitor the changing physical context. This physical context 
affects the information exchanges -interspersed with talk about evaluations 
or anticipations, are comments about events and data. For example, notice 
how the attending makes reference to the urine output level as he is talking 
about management activities. So even though team members are talking 
about management and evaluation, for example, they may also talk about 
perceived events occurring in the present. 

This physical context also plays an important role in facilitating 
communication (See section on Contexts for Communication) and in the 
observability of interactions (see section on Information through Noticing) 
Figure lib has an example of the latter; towards the end of the episode the ’ 
attending sees that the resident is writing down the lab results as he gets them 
over the phone. The attending is able to look over the resident's shoulder 
and read the relevant data as soon as it comes in (i.e., to be able to take the 
management action as soon as possible.) 

Figure 1 1 provides another example in which team members discuss 
and evaluate a course of management taken earlier in the case. 

These episodes illustrate the "on-going talk" about the monitored 
process and problem solving state that team members engage in. This is a way 
team members can keep one another calibrated in the moment-to-moment 
interpretation and management of the case. 
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TRANSCRIPT DOMAIN INDEPENDENT 

DESCRIPTION 


[212:15:00] 

RJ: I did another output 1 and it was five four, 
something like that 

RS: So she likes the dobutamine^ 

RJ: Her SVR ® came down (8 point 2) 

RS: So she he likes-that’s-we could come 

back down on the nitro^, come down about a 
half if you want (RJ turns it down on infusion 
device} 

RS: I’m gonna get some more gloves. I'll be 
right back {RJ steps away} 

(2 minutes later RS returns} 

RJ: (points at temperature indication on v.s. 
monitor }: .. cold. .hypothermia.... 

RS: I think the problem was I dumped all 
that mannitol^ in, boom (chuckles} ...well 
it’s just— her blood warmer is on right? 

RJ: yeah, I checked that 

[ omitted utterances about blood warmer] 

RS: Turning down the nitro was a good thing 

to do, she’s starting to get I think the 

dobutamine’s finally done its job. Took it a 
while. 


RJ informs about result of test 
RS evaluates result 

RJ informs of relevant parameter, provides 
reference and value 

RS suggests management action 

RJ takes suggested action 


RJ queries about parameter value 

RS provides possible explanation involving an 
intervention he took 

RS queries about relevant equipment state 
RJ responds: already checked 

RS evaluates previous intervention on process 

RJ concurs 


RJ: ...right back where she was 

^Test of cardiac output. 

^Generally recommended in cases of increased SVR, normal heart rate and low cardiac output. 
^Systemic vascular resistance, which is blood pressure divided by cardiac output. 

^Nitroglycerin, a vasodilator, for controlled hypotension, useful in patients with known or suspected 
coronary disease. 

^An osmotic diuretic given intravenously. 

Figure 9, Episode: Evaluating the effects of interventions. 



TRANSCRIPT 


Domain-Independent 
Description 


[ 113 : 23 : 40 ] 

R: I'm gonna go ahead and send 
another gas I 

A: yeah, let's send another gas 
and 

R: see where we're at. Have a 
feeling it's^ still gonna be low, he's 
just oozing all over the place 3 

A: I think once we bring the 
temperature up we have done all 
we can do, you know, he's putting 
out urine 4 1 think I see more there. 

R: yeah, there is a little more there. 
I'm gonna empty that in a couple 
of minutes 

A: yeah. (..) Dr [R] 

R: temporarily yeah 

A: I mean, there is nothing we can 
do other than Bear Hugger™ now. 
We can get one more and put it on 
the lower part of the body. 

Another Bear Hugger™, we can 
get some heating lights on the 
field. 3 


R informs A of measurement he plans 
to take 

A concurs 


R states expectation for measurement 
value and reason 


A states concern for particular 
parameter, summarizes state of 
management : “we’ve done all we 
can do" A states an observation: 
parameter that has been of concern 
seems better 

R verifies, mentions taking an action 
that will allow them to assess value 


A summarizes management plan 
concerning one parameter: provides 
two options 


[blood gas taken; episode continues] 


Send J ng a blood 8 as refers t0 sending a sample of blood for analysis of- dH 

press, ‘ re ° f 1 hematocrit, lie excej, Sodium, 

2 Hematocrit 

reference to the patient's bleeding. 

5 ^ ow urine level has been a concern so far in this case. 

All of these options concern efforts to maintain adequate temperature. 

Figure 10. The state of management 


Figure 10 (continued). 


TRANSCRIPT 

{Blood gas taken 9 minutes ago) 
[continued] 

A: we'll see how the blood 
hematocrit comes back, if it is low 
then we'll add a unit of blood. I'd 
rather more blood than crystaloids 

R: I think W s gonna be low. I bet 
you it'll be 25, maybe 28 (they 
look at surgeons) 

A: are you gonna be taking any 
grafts also or just 

S: oh yeah, man we're gonna be 
we're gonna you name it this is a 
bear, yeah we gotta we're gonna 
give him some bone so maybe he'll 
fuse, heal 

(Phone rings, [RJ answers) 

R: OR [#] (patient name) umhm, 
umhm, (A looks over for a moment 
to area where R is writing, then looks 
up) 

A: [N], I'll take 2 units of blood. 

R: 1.4, 8,4 (pause) alright, thank 
you (hangs up) 

A: I think ...should be air and 02 
only (adjusts knobs) he's not liking 
nitrous very much 

A: What was the calcium? (looks 
at record on table) 

R: It was down alittle bit, 1.84 

A: I'd give him 500... (R gets up) 


Domain Independent 
Description 


A informs of plan, dependent on test 
result value 


R states expectation 


A requests information from different 
team member which could affect 
course of management 


A looks at incoming result 


A directs another team member 


A takes action, informs and provides 
explanation 


A requests parameter value 

R responds, provides reference 
information 

A directs management action 
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TRANSCRIPT 

/5 1 2:54:43 7 

A: Something I can help ya with? 
RS: Nothing, he's doing okay. 

A: Did you get an output 
recently? 

RS: {turning vs. knob} yeah, 7,9, 
let's see 
A: really!? 

RS: yeah. That was a combination 
of 3 outputs so it's pretty 

accurate. His index 1 is 3,4. It's 
still low but I'm just 
A: .1 would just 
..LR.2.. otherwise... 

RS: 1 11 put a little LR up on there 
(indicates to left N tree}, I’d rather.. 

[a few omitted utterances about 
gas results] 

RS: yeah {pause} I think we did the 
right thing, I think things went 

really nice this morning. 3 
A: you can never be faulted for 
over-monitoring somebody. I'm 
sorry. 

RS: Prone position just adds 
another factor that you have to 
think about. 


Domain Independent 
Description 

A offers general assistance 

A requests information on a parameter 
of concern 

R informs of value 

R elaborates with information about 
accuracy of value, informs of related 
parameter value 

A suggests management action 

R indicates he will take it, elaborates 


R evaluates past action/strategy 

A confirms, elaborates 

futher confirming and elaborating of 
past action/strategy 


1 Cardiac index, which is the cardiac output divided by the surface area. It allows one to adjust 
2° ^ n 8 Pat ' ent: ,h8 CafdiaC ° Utput Sh0uld 9 0n0ra »y be larger for larger people 

tj iv ,iuw c °- ai " s ■ — - - - * ~ - 

l A re,eren0e t0 ‘be decision to place a Swan-Ganz catheter earlier in the case At the 
diseased C3S6 th8r0 W3S ^ U " Cer1ainty about the exte "t 01 ‘he patient's heart 


Figure 11. Evaluating a course of action. 
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Explanations for Interpretations and Actions 

In the transcripts many explanations for interpretations and actions 
have two basic characteristics. One is that they are unprompted that is, they 
are provided along with the stated interpretation or action, rather than being 
given in response to a query. Note that this term is not meant to imply that 
there is no external stimulus for the explanations in the environment, but 
simply that they are not explicitly requested by another team member. The 
other characteristic is that they are often brief and tend not to be "deep level" 
explanations (i.e., they do not make domain knowledge explicit). In the 
terminology of Toulmin (1958), team members often provide grounds (the 
data on which a claim is made) rather than warrants (the relationship 
between the grounds and the claim). The example below illustrate both 
characteristics. 

In the following interaction, the attending has been talking to the 
resident about non-case related domain knowledge, then without pause he 
comments on something he has noticed: 

A: .... He probably needs some fluid I would think, his urine looks pretty 
dark. 3 

RS: Yeah, let's give him some of this. 

In this example, the attending tags his assessment (which is also a 
directive in this context) with an unprompted explanation ("his urine's pretty 
dark"). Note also, that stating the data on which the assessment is based, 
evidently suffices as an explanation. For this to be possible, there needs to be 
shared mutual knowledge (i.e., such explanations would be cryptic to a 
layperson), in some cases it may rely on mutual knowledge established 
during the case. Another way to put this is that explanation can be "compact" 
because it relies on various contexts (see section on Contexts for 
Communications.) 

Communications of actions are also sometimes coupled with brief 
explanations for these actions. For example, when the attending in Figure 10 
turns down the nitrous oxide, he informs the resident of his action ("..should 
be air and 02 only" and also supplies the explanation ("...he’s not liking 
nitrous very much. ) As in the above example, this brief, cryptic explanation 
requires mutual knowledge to be completely understood. 

Not all of the spontaneously offered explanations will be needed from 
the listener s point of view. However, the general tendency to provide 
unprompted explanations is useful because it is a mechanism to add to the 
common ground and thereby forestall future misunderstandings. 

Long explanations of reasoning are the exception rather than the rule 
in episodes of management and diagnosis among team members. Where one 
sees long, retrospective explanations of reasoning are in tutorial situations 
and when there are disagreements about a course of action. Tutorial 
explanations, which are about the subject matter and practice of 



transcript 

[1010:40:001 


Domain Independent 
Description 


R: you know we're not gonna be able to get 
anything bigger than a 7-0 down his nose 

A; that's fine.... 


A. [R2] let's draw up some, grab a lOcc 
syringe, and a 16 gauge IV catheter, take 
the needle out of the catheter. And then 
draw this stuff up (hands object to R2j he 
does have tight nostrils. 


R. I just think an oral's the way to go 


A: I'll tell you why later, draw it up just 
from the syringe then we'll just drip that 
down this nostrils first... 


A defers explanation (for course 
action) 


R: ,t0 we need a second IV and we 

need an A-line we haven't started on yet 
R: I'm gonna start on the A-line, {coes to 
P s left arm } 


[omitted utterances] 


[1010:44:351 

R: I think topicalizing with that is the 
way to go, I think it's the best there is 


A: Oh I agree, but I still believe that 
(first medicine) was absolutely 
necessary. I'll explain why 
when(...)experiences on Thursday. 

R: Oh I heard about that, I think I 
walked through while you were doing 
that ° 


A provides partial explanation 


A: So, I'm just gonna be very conservative, 
[continued! 


Figure 12. Deferring Explanation. 


Figure 12 (continued), 
TRANSCRIPT 
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Domain Independent 
Description 

[R has difficulty doing the nasal fiberoptic 
intubation ; a new scope is brought in; finally A 
tries it] 

[1011:13:15] 

A: Deep breath (to P } 

R: Let's just try an oral just once, see if we can do it 

A: Deep breath (A inserts tube} pressure (.) deep 
breath 


R: ..think he got it 

{A listens for sounds of correct placement} 

A: See [R]. (then to P} Alright you're gonna go to 

sleep now I haven't lost my touch. We're going 

to sleep now (A injects P, R squeezes ventilator 

bag} Deferment of explanation (for course 

of action) 

A: When he's asleep I'll tell you why I didn't do 
an oral 


(R adjusts anesthetic agent} 

R: Okay, is that what you tried to do the other 
day? you had to.... 

A: We tried everything. This has to be jammed in 
a little tighter, (referring to some apparatus 
connected to P} 


[1011:15:08] 

A: ..some agent? 

R: yeah 

A: I just think with orals you really have to 
sedate him a lot more... 


A provides explanation 


R: See I don’t. I’ve done enough of them, you don't 
have to sedate them at all 

A: (..) that big airway... 

R: Absolutely. 

A: But you should be able to do them both 

R: No, I just think, my choice with a guy like this 
with very small nares, and he’s this large, I 
rather have a bigger tube, and to do that you 
have to go oral 
(R adjusts ventilator settings} 

A: I think a 7 will be no problem with him, we'll 
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anesthesiology, 1 are intended to help a less experienced practitioner (typically 
a medical student or sometimes a resident) gain domain knowledge In the 7 
teaching setting where the data were gathered, teaching and learning are part 
of the work environment: residency training is an apprenticeship and ? 
medical students are allowed to attend and participate. 

The following episode illustrates the important role that the on-goine 
events and that the tempo of activity have in affecting the timing of 
potentially long explanations (See Figure 12). The explanation in question is 

thTnt h ? 1V6 2 b6 ? USe there is a disa g re ement about a course^f action- 

would TaS Hn S deCld ^ ° n a nasal fiberoptic intubation, but the resident 
would rather do an oral fiberoptic intubation. The resident voices his 

opposition but the attending has final say. Note that at least twice the 
a en mg defers his explanation for the course of action. This deferment is 
ased at least m part on the time pressures or on the situation not being 
appropriate h is only after patient has been intubated and anesthetized that 
the explanation is finally completed, and the topic is dropped. 

Unprompted Communications 

Information Ex changes about Activities 

Prac’fi , There are numerous physical actions required to perform anesthesia, 
ctitioners engage m many different activities in order to get the patient 

^pf|prhpH r an ft Sth if S1 K 3 | nd then . as P art of the maintenance of anesthesia. This is 
eflected in the high degree of coordinative talk, which includes explicit 

escnptions of one's own activities and queries about the activities of others 
This is most obvious in tasks that requires fine synchronization, such as 
urmng a patient on the operating table. Team members inform others of 
mished activity and seek information about such activity. This is important 
because many activities are contingent on a particular phase of the operation 
or on existing preconditions (i.e., other actions having been taken by other 

TZT iT T n °u been t3ken by ° thersX For exam ple, adequate anesthesia 

estahlkhS h ° re V 5Urge0n beginS incision - If this formation has not be 
established by some other means, the surgeon will typically ask the 

anesthesiologist some variant of: "is it okay to begin?" Note that this 

coordinative question also serves to inform the anesthesiologist that the 
surgeon is ready to begin. 6 

Distributed management and diagnosis requires that team members be 
aware of those activities of others that may affect the monitored process 
Keeping track of the various potential influences on the process is critical for 

•_“f n ° sis order t0 kn ®w what may be the cause(s) of an anomaly). It is also 

mportant for management because knowing what to do depends in part on 
knowing what has been done and what is expected. 


1 For example, they can be about 
drug) or about how to perform a 


"how something works" (e.g., the cardiovascular response to 
particular technique, or about how some equipment operates. 


particular 


43 


Team members assist one another in this respect by unprompted 
communications about their relevant activities. The transcripts show many 
instances of team members spontaneously telling one another what they 
have done, are doing or are about to do. The following example illustrates a 
case in which a team member informs several other team members about an 

ac ivity he is about to undertake that may affect their current or planned 
activities: r 


[ 510 : 26 : 35 ] 

(RS brings a tray on a movable stand to P’s left side , sets a sterile "kit" 
on it; someone else stands a couple of feet from tray} 

RS: Okay, I'm gonna be opening up a kit here so just watch your elbows 
{opens it up} 3 

In some cases a junior member will inform a more senior member of 
w at he is about to do. For example, the junior resident sometimes lets the 
senior resident know that he is turning on the anesthetic agent or that he is 
going to be giving a drug. These "informs" of activity have another purpose 
bes des letting the team member know about a new influence on the process* 
it allows for confirmation, prevention or modification of the planned action! 
The following episode, which occurs just before induction, illustrates: 

[ 810 : 48 : 40 ] 

R IsenT™ tUfn ° n ^ St 3 1Utle F ° rane ' since {moves Awards anesthetic 

A: No 

R: No? 

A: I mean you have (syringes in your hand )... 2 

i , , G ^ erall y ^ough, one finds that actions are approved (see example 
elow). This is to be expected given the informing team members are 
relatively knowledgeable, experienced and grounded. 

[ 212 : 31 : 35 ] 

RJ: I m gonna turn the nitrous back on now 

RS: yeah... 

Not only do team members inform one another about their own 
relevant activities, they may also inform one another about a relevant action 
taken by a third team member. In the following episode, the information 

assists m preventing an undesirable situation (administration of a second 
dose of a drug): 
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[210:50:33] 

RS: Why don't we try a little ephedrine on her (RJ turns knob on v.s. 
monitor, the left upper window goes away, gets complete waveforms} 
RJ: Yeah, he just gave some ephedrine. 

RS: Did he? Okay. 


Highlighting anomalies, events and parameters of concern 

When a player is a particularly high scorer it is worth noting 

periodically how he is doing. " 

— So you want to be a Sportscaster. Coleman, Ken (1973). 

Besides unprompted communication of activities, team members 
point out anomalies and events to one another, they voice concerns and talk 
about evaluations of past interventions or plans. Particular parameters may 
become especially important to monitor either because of some preexisting 
patient condition or because they become anomalous during the case. The 
quotation above conveys the importance of monitoring and communicating 
about certain parameters of concern, even when they are not necessarily 
anomalous. For example, during updates, team members will often comment 
on the parameter of concern whether it is abnormal or not; that a parameter 
of concern has become normal is also informative. 

The following episode illustrates how drawing attention to a parameter 
of concern can lead to a discussion about management that both evaluates 
past management actions and attempts to develop new ones. This episode 
comes from a case involving a patient who has lost a lot of blood prior to this 
episode. His temperature became a parameter of concern early on, as did his 
urine output. 

[1 1 1:43:44] 

A: {looking at the v.s. monitor} temperature 35.2 eh? 

R: yeah, I turned the room temperature back up but 
A: but 

R: I think it slipped down somehow 
A: oh 

R: I turned it up when I came in 
A: running.. ..and the humidifier 
R: got the humidifier on maximum 

A: the other thing we can do that has helped is have that aluminum foil, 
sometimes we can just wrap the circuit this way the loss of heat by 
radiation is less and it kind of just keeps it little warmer {looking at 
monitor) but I think since now they have covered the field {indicates 
back}.. hopefully it should be. But ah 

R: I thought these heating wires were supposed to keep it warm 
A: supposed to, but you know 



R: anything helps 

A: anything helps just a little less heat loss here and just alittle less heat 
loss there 
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An hour and a half later the attending makes another reference to the 
temperature. 

[113:3:40] 

{ A and R looking at v.s. monitor } 

A: temperature... 

R: I've been turning the room temp up, there's not much more we can do 
unless we get, they don't have any of those um, one time, they 
demonstrated a Bear Hugger that could be used interop? 

A: yeah,.. bring the Bear Hugger™, you know 

R: {gets on phone} could we have one interop Bear Hugger™.. 

Pointing out anomalies and parameters of concern to one another may 
lead to a discussion about management in which new ideas may be generated, 
as in the above segment. It can also serve to make another team member 
aware of something about the process that another team member believes 
significant, and which they may not have noticed. 

Relative References 

An important point about how team members talk about anomalies 
and parameters of concern is that they often talk about them in a relative 
way, with reference to what the parameter value was earlier. For example, in 
episode 9, the resident states that "her SVR came down.” In episode 10, the 
attending asks about the calcium and the resident states that it was "down a 
little bit , 1.84." 

Information through Noticing 

Much information about the state of problem solving can be picked up 
by overhearing and seeing what other team members are doing; it is not 
necessary for team members to always direct attention and explicitly provide 
this information to one another. Valuable information about team members' 
intentions and potential influences on the process can be picked up by being 
able to observe the activities and interactions of others. In studying cockpit 
crews, Segal (1994) notes the importance for coordination of being able to 
observe the activities of other team members (a form of information 
exchange which he terms consequential communication.) Some work 
environments foster "open" interactions, i.e., interactions that are observable 
and understandable by others (Huchins, 1990). Aspects of work 
environments also allow team members to gain information about activities 
that have already been taken. For example, anesthesiologists can tell what 
drugs other team members have given by looking at the anesthesia record, or 
by seeing what ampules are empty on the drug cart. Assessments and plans 
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may also be picked up or inferred by overhearing interactions between other 
team members. All of this information is ultimately valuable in allowing 
team members to update their situation assessment and expectations of the 
monitored process. 

Team members notice what others are doing and on relatively rare 
occasions may observe behavior that they don't understand, that doesn't fit 
with their expectations, or that suggests that the other team member could 
use assistance. They have a sense of how activity should be occurring and are 
able to pick up on discrepancies in the expected activity -when things seem 
unusual." It is generally in such instances that a team member questions 
another about his activity. See episode below. Open interactions allow 
possibilities for individuals to detect actions that may be inappropriate in 
context and to initiate recovery before outcome failures occur. 


[4 \ 0:39:25} 

(RS sprays numbing medication into P's mouth, turns to get gloves, 

^ iS P s ie f f arm sli ghtly, touching pressure cuff line} 

RS: What are you looking for? 

RJ: Just to see if that was {points towards monitor} correlating with 
that. 3 


RS: {looks towards monitor, putting on gloves} They were correlating 

yeah, very well, {looks back to P} She's just a little anxious with me 
doing this. 


Another notable source of information is the "self-talk" of others. 

Team members occasionally talk aloud when engaged in a task or when 
trying to figure something out. This may serve a dual purpose. First of all it 
may help the practitioner who is talking to "keep track" (e.g., of required ' 
actions or possible alternatives.) It is also a mechanism that allows other team 
members to notice someone's activities, plans or reasoning and to provide 
assistance, if necessary. This is accomplished generally without distracting or 
demanding attention. A common, brief form of self-talk which is found in 
the transcripts is saying (or sometimes sighing) "okay" or "alright" upon 
completion of a task or subtask. This can, in some circumstances, assist in 
finished 11118 behavi ° r by lettin S someone else know that a particular stage is 

j ^ eat ^ ail d Luff (1991) point out that the coordination in the London 
underground line control room relies on "surreptitious" monitoring of self- 
talk by other team members. Their field study found that it was "relatively 
unusual" for team members to explicitly give information to one another. 
Overhearing and monitoring actions allows them to keep aware of what 
others are doing. The structures of certain work environments coupled with 


, measuremen * as ; indicted by the arterial catheter and that measured by the 
b&SZSSh 0n the « rterial Une Woodpressure measurement is done intially 
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the demands and tasks performed may make it unnecessary for team 
members to explicitly tell one another about their actions and assessments 
even though coordination is required. 4 

Shared Tools 

Information about the state of the process and of problem solving can 
also be picked up from the tools that are publicly available to the team 
members, such as the various displays and the anesthesia record. In order to 
maintain a common frame of reference, these public tools need to afford 
information access in a way that is consistent with all team member 
expectations. Consider a shared artifact like the anesthesia record. It is used by 
several people both for recording actions and values (e.g., what drug was 
given when and how much, or what the blood pressure has been for 5 
minute intervals throughout the case) as well as for retrieving that 
information. A representation that is used and modified by team members in 
different ways can create rifts in the common ground. In the following 
episode, the senior resident comments about the different way the junior 
resident has annotated the record. The junior resident has written the results 
of the blood gas sample which was drawn at 1:45 on the first page of the 
anesthesia record. However, the last time written across the top of the first 
page is 12:30 (see Figures 13a and b). 

[10 1 7:56:40] 

{The senior resident has just returned from a break. The junior resident 
who remained in the operating room begins to update him, while the 
senior resident looks at the record .} 

RS: {looking at record} oh, you just drew another gas 
RJ: yeah, I just sent 

RS: I usually end up putting the next gas, when you go to a new page 
over here so you can look down 
RJ: oh so you can follow it 

RS: it's not a big issue, that's what I usually do. No big deal. 


Humans have mechanisms that allow them to become aware of stimuli that are not directly attended to, e.g., 
peripheral vision, sensitivity in divided and focused attention tasks to important words, and to changes in 
volume or pitch. This is exploited in domains that use voice loops (e.g., in mission control or aircraft carrier 
operations). Rochlin et al., (1987) points out that in aircraft carrier operations checks are routinely 
performed on decisions via a constant conversation loop. Rochlin et al. say "At first little of this chatter seems 
coherent, let alone substantive, to the outside observer. With experience one discovers that seasoned 
personnel do not listen' so much as monitor for deviations, reacting almost instantaneously to anything that 
does not fit their expectations of the correct routine." 
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Queries and Informative Responses 

Team members sometimes request specific information about events 
or parameters, such as "What is the blood pressure?" as a surgeon might ask 
during an aneurysm clipping surgery. But other questions are more open- 
ended, such as some variant of "What's up?" which attendings may ask upon 
returning to the operating room. Some queries fall somewhere in the middle. 
An observation statement such as "temperature 35.2, eh?" in the context of an 
earlier episode, besides being a comment on the temperature being low, is 
interpreted by the resident to mean something like "tell me what you know 
about this parameter being low." 

The point is that team members don't have to explicitly query about all 
the information they may need, because team members can respond 
appropriately to open-ended questions. This is important in the case of 
tutoring, when learners lack knowledge about what information they are 
missing (e.g., Miyake and Norman, 1979), but it is also relevant in updating 
situations. Indeed, even when team members generate specific queries about 
the process, an assessment or action, responders often do not simply answer 
the explicit question posed to them. They "go beyond" the question to provide 
what they deem an informative response. People are sensitive to the 
intentions and goals that requesters have when asking for information, and 
they answer accordingly (e.g.. Pollack, Hirschberg and Webber, 1982). People 
can provide relevant information even though an information seeker does 
not formulate the question precisely. 

One way that team members go beyond a minimal response is by 
elaborating. For example, consider the following question and answer pair: 

Q: What is the blood pressure? 

A: It's 110 over 60. 

The answer given here might be called a minimal response ; it answers 
the explicit question posed. But, it may be informative depending in some 
circumstances, to provide an answer that goes beyond a minimal response. 

For example, it may be important for the information seeker to find out about 
trend information; if the blood pressure has been highly erratic recently, or it 
is expected to go vary due to a recent action whose effect is not yet apparent. 
Knowing these factors may be useful and even critical to the team member 
requesting the information. 

Consider the following example from a surgery involving a cerebral 
aneurysm clipping. Before the clipping, deliberately induced hypotension is 
generally used in order to minimize the chances for rupture, facilitate 
placement of the clip and also to reduce blood loss if bleeding occurs (Barash, 
Cullen and Stoelting, 1991). The anesthesiologist, because he is the team 
member who administers the drugs, must coordinate with the surgeon 
concerning the start, duration and degree of hypotension. Figure 14 shows 
two instances in which the surgeon asked the anesthesiologist about the 
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blood pressure: right before the clipping and shortly after the clipping. The 
figure also shows the desired blood pressure values around the time of 
clipping. In the first asking instance, the anesthesiologist provides a minimal 
response plus a “tag" that informs the surgeon that the value should soon 
drop to the expected value. After the clipping, when the pressure is to be 
brought up again, almost the same minimal response is provided, along with 
a tag that, this time, specifies how the value is related to the normal value. 
This example illustrates that team members provide a more "complete 
picture" for the information seeker than that which would be provided by 
simply answering their explicit question. In answering queries about process 
data, providing an informative response means providing information about 
factors that will or might affect the value within a certain horizon of the 
future. 

The ability to provide responses that are sensitive to the goals of the 
information seekers is important. 

Another example of a unprompted elaboration is found in Figure 11. 
The attending seems surprised by the parameter value told to him by the 
resident. The resident's response is to provide further information about the 
accuracy of the cardiac output (i.e., “that was a combination of 3 outputs so it's 
pretty accurate"). He also provides information about another parameter 
value that is related to the one asked about (i.e., cardiac index). Such 
elaborations are commonly found in the data. 

Surgeon: "What's the blood pressure now?" 
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Another way that team members go beyond a minimal response is bv 
providing responses that vary on the dimension of interpretability of process 
data. At one end there are statements of data values and at another end are 
interpretations of what they mean. Consider the following exchange: 

[8 1 2:24:20] 

S: Did he get a good diuresis? 

R: I just emptied uh 350ccs or less, 25 minutes ago. 

S: Huh, did he get a good diuresis? 

R: 350 cc's (.) half hour. 

In this case, the surgeon has on the surface of it asked a qualitative 
question. That is, the form of the question implies a judgement (e.g., was it 

good or not ). However, the responder chooses to answer it quantitatively 
in effect leaving the judgement to the surgeon. 4 Y ' 

Updating the Common Ground When a Team Member Returns 

The trend in complex, dynamic fault management domains is to have 
fewer humans monitoring interacting subprocesses through the increased use 
intelligent systems. Consequently, the human may be distanced from the 
process for relatively long periods of time. What is the nature of effective 
updates. The following episode is particularly interesting because the resident 
and attending engage in joint diagnosis. 

Overview of HpisoHp 

The episode occurs during the maintenance phase of an operation to 
chp a cerebral aneurysm. The episode occurs about an hour after induction 
and before the surgeons have exposed the aneurysm. The senior resident is 
the only anesthesiologist present; the attending has been away for about half 
an hour and the junior resident is on break. In this episode the senior 
resident detects an anomaly - bradycardia (very low heart rate). He takes 
corrective action by administering atropine, a drug that raises the heart rate. 

He has the attending paged. He mentions the event to the surgeons and 
enquires whether they "might have been doing anything." They answer no. 

diagnosis^” 8 amVeS after a few minutes and together they arrive at a 

Anomaly Detection, Corrective A c tion and Investigation 

To a practitioner, the bradycardia event is quite dramatic. The pulse 
rate as indicated by the beeping of the pulse oximeter suddenly slows down 
The resident, who has bent down (apparently to check the urine output or to 
begin a cardiac output measurement), immediately gets up to look at the 
monitor. Five seconds later he injects the atropine. See Figure 15. 
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Bradycardia may be expected in certain situations. For example, certain 
rugs given during maintenance can result in a lower than normal heart 
rate. Also, a low heart rate indication could be expected in the case of a 
loiown artifact with monitoring equipment. However, in this case, 
bradycardia of such severity is unexpected. Because of its severity it is critical 
to treat it immediately, before its consequences begin to propagate. It is also 
important to understand its etiology because it could be an premonitory 
event, i.e., indicative of a fault that needs to be managed or corrected to 
prevent the condition from recurring or to prevent other possible 
disturbances. After investigating the surgeons' actions as a source of the 
event, the resident pages the attending to help him uncover the cause and 
also to make the attending aware of a potential premonitory event. 

Update and Toint Problem Solving 

f 11 f / gUr L 6 16 sh ° ws what occurs when the attending arrives. Notice, first 
rffh' t | he . resident answers the attending^ open-ended query with a 
ather detailed account that includes several pieces of information. One of 
these is a related process event (less severe bradycardia) that occurred before 
he severe event. Notice that he provides information about the dynamics of 
the antecedent event, of the event itself, and of another relevant parameter 
(blood pressure ). 6 He mentions what action he was taking on the process 
while the event occurred, the dynamics of the event, the limiting values 
reached and the corrective action he took and the process response to it 
Finally he informs the attending about the state of problem solving, i.e. that 
he has no explanation. He has rejected one hypothesis (i.e., "nothing [the 
surgeons] were doing"), though he doesn't elaborate. At the end of the initial 
update, the attending queries him on this point. The resident's response is 
the same, unelaborated. v 

Notice the form of the initial update. The resident recounts it like a 
story, basically preserving the order of events. Such a recounting would seem 
to benefit causal analysis. 6 

At this point, the state of problem solving seems to reach an impasse 
(i.e., when the attending says that he "can't necessarily explain that.") 

However, the resident continues the problem solving by telling the attending 
about various management influences on the process (i.e., drugs being given). 

e en engages in hypothesis discounting—mentioning a few potential 
(incomplete) hypotheses and providing reasons for discounting them. 

The attending then lists what the causes of this kind of event have 
been m his past experience. In reaction to this, the resident seems to 
reevaluate the data that fed into his conclusion that it could not have been 
due to the surgeons. He "revisits" what, based on the attending's knowledge 
seems to be an important time frame. He then point out in detail what was ' 


^Halothane or large doses of morphine or fentanyl (Chung and Lam 1990) 

rS ou t h tff s rec n £sm ay “ by 3 reflex b * the absence «* high blood pressure 


55 


occurring then— that it was actually when the surgeons were engaged in an 
activity that could have given rise to the event. 

This example is used, not to suggest that this particular update is 
optimal in content or pattern. Rather, we use it to illustrate some 
characteristics of cooperative problem solving. One point is that diagnosis can 
be collaborative and cooperative rather than autonomous. In this episode the 
resident has access to the relevant data by having been present during the 
event, while the attending has access to more etiological knowledge. Both are 
essential for the appropriate diagnosis in this case. 

Another characteristic illustrated by this problem solving episode is 
what one might call "robustness." It is robust in the sense that an initially 
discarded hypothesis is reintroduced and taken as the best explanation for the 
event. One aspect of the team interaction that seems important to its 
robustness is the ability of the resident to reexamine past data in the light of 
the attending's concerns. 

Some implications for cooperative interaction in supervisor updates 
are suggested by this example. One is that an important characteristic of an 
intelligent subordinate team member is to be able to recognize that the 
situation is in danger of escalating beyond his or her (or its) competence, i.e., 
knowing when to call the supervisor. Secondly, the subordinate must be able 
to provide some kind of reconstruction of the event that emphasizes relevant 
events, actions and relationships in order to provide the supervisor with a 
coherent recounting of the events that led to the present state. 
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TRANSCRIPT DOMAIN INDEPENDENT PHASE OF PROBLEM 

DESCRIPTION SOLVING 

A: (enters room} Nice and A comments on process 
tachycardic^ 

R: Yeah, well, better than 
nice and bradycardic 


A: What's going on guys? A makes open-ended 

request for update 


R: (takes end of printout, 
seems to show to A} She 
had an episode of just 
kinda, all of the sudden 
bradying down to 50, 52 
then came right back up, 
nothing they were doing, 
then all of the sudden out 
of the blue, I was shooting 
an output^and she dropped 
down to 32, 38^ somewhere 
around there, pressure** 
dropped down to 60 so I 
gave her .5 of atropine^ 
and ah, kicked her up to 
6.5; she liked that, but no 
explanation. This is at 50 
millimeters per second, 
twice the speed^. 


R mentions; 

-previous related event, 
including dynamics and 
approx values 

-discounting of other agents’ 
activities as cause 

-action taken while event 
occurred 

- dynamics and approx values 
of relevant parameter 
during event 

-corrective action taken and 
process’ response 

-no good candidate for 
diagnostic search 

R supplements description 
with artifact preserving data 
history 


Initial update of 
significant event 


A: They weren't in the head A requests specific past Hypothesis building 

doing anything? observation information 

(concerning other agents’ 
activities) at time of event. 


[continues! 

^Tachycardia refers to rapid heart rate, while bradycardia refers to a slow heart rate. 

^ Cardiac output refers to the volume of blood per unit time that the heart moves. The measurement of cardiac 
output requires injection of a measured amount of IV fluid and is done infrequently. 

^ These are very low heart rate values, requiring treatment 
4 blood pressure. 

A drug that increases heart rate by blocking the parasympathetic system. 

^ Chart speed for EKG recording is usually 25 mm/sec. Because it's running at 50mm/sec, recorded events 
occupy twice the length of chart paper than they would at normal speed. 


Figure 16 . Bradycardia Update. 
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Figure 16 (continued), 

TRANSCRIPT DOMAIN INDEPENDENT 

DESCRIPTION 


R: Nothing. 


A: Okay. Well I can't 
necessarily 

R: The only thing 

A: I can’t necessarily explain 
that 

R: Yeah, neither can I. The 
only thing we’re doing 
right now is just trying to 
open her up and fill her up. 
(points to right IV tree} 
She's up to a mic per kilo of 

nitro^ and then she's still 
at the 5, started out at 3 

and a half of dobutamine® 
and it did absolutely 
nothing, so I’m up to 5 

A: Okay 


R answer discounts 
hypothesis, but does not 
elaborate. 


A states has no candidates 


R provides more information 
on current actions and 
previous actions 


[continues] 


PHASE OF PROBLEM 
SOLVING 


Context building 


^Nitroglycerine. A vasodilator, for controlled hypotension. 

® Dobutamine is generally given for low cardiac output, in order to increase contractility. 
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Figure 16 (continued), 

TRANSCRIPT 


R: So I don't know if she 
doesn't like contractility 
or, I can't think of 
anything else we're doing. 
The line went in perfectly 
normal, I can't imagine 
that she has a pneumo or 
anything that would be 
causing tension, her peak 
area pressures have not 
changed. Just all of the 
sudden -boom-out of the 
blue-her potassium is 3 
point 3 and we're getting 
ready to replace that and 
we have been hyper- 
ventilating, but I don't 
know if low potassium can 
affect heart rate 

A: Yeah, I don't know, I can’t 
give you cause and effect on 
that. In my experience it's 
usually been stimulation of 
the trachea, it's something 
traction on the dura 

R: yeah, (absolutely) 

A: you know things 

R: yeah, it may have been 
dura 

A: ...sort of a reflex, pressure 
on an eye 

R: { animated } Actually it 
was when they were 
sawing the dura open. 


DOMAIN INDEPENDENT 
DESCRIPTION 


R offers hypothesis but 
discounts based on his 
knowledge 

R offers another hypothesis 
but discounts it based on 
data 


Dynamics of event repeated 

Process variables mentioned, 
action to be taken 
mentioned 

R offers a third hypothesis but 
voices his lack of 
knowledge 


A mentions two causes of the 
significant event based on 
his past experience 


R remarks that one of these 
causes may have been 
cause in this case 

A provides another possible 
cause based on past cases 

R remarks that event occured 
during a time when one of 
the causes mentioned by A 
could have occured 


A: well that's 
[continues] 


PHASE OF PROBLEM 

SOLVING 

Hypothesis discounting 


Case-based discussion 


Discounted hypothesis 
reconsidered 
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Figure 16 (continued), 

TRANSCRIPT 


R: putting tension on it 

R2: traction on the dura 

A: you touch the dura you'll 
get that 

R: okay 

A: cause the dura is 
ennervated by the fifth I 
believe, and it somehow 
makes its way back to the 
(.) ganglion, same thing 
that causes oculocardiac 
reflex 

R: I'd be willing to bet you're 
absolutely right (RS waves 
pen over ventilator setting 
knobs , then leaves view) 

A: is the same mechanism 
whereby you get 
(bradycardial traction) on 
the dura, so my guess is 
that's exactly what it was 

R: Okay. 

A: you now and for future 
reference, if you suspect 
(pause) this lady's 
probably not going to mind 
this experience because 
she, we don't think she's 
really significantly sick, 
we're being a little overly 
cautious with her, my 
preference is, if you have a 
patient that you think has 
a bad heart, and you think 
they have a vagal problem 
via traction, or an eye... 

RS: so that's why 

A: It's traction on the dura 


DOMAIN INDEPENDENT 
DESCRIPTION 


A states mechanism 


A describes mechanism 
whereby hypothesized 
cause leads to the 
significant event 


R expresses confidence for 
hypothesis 


A continues explanation of 
mechanism 


R concurs (with hypothesis) 


PHASE OF PROBLEM 
SOLVING 


Hypothesis 

acceptance 
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CHAPTER V 
DISCUSSION 

It has been a consistent observation that advisory systems for dynamic 
fault management provide diagnoses and explanations that congregate at 
busy times, creating extra tasks and cognitive burdens for operators (Woods et 
al-, 1991; Malin et ah, 1991). This field study provides a clearer picture of why 
this is so: it indicates that human team members, by contrast, support one 
another in maintaining an updated understanding of the evolving situation. 

The term “clumsy explanation" was used to refer to a form of 
diagnostic support that is not well adapted to the cognitive demands of 
dynamic fault management. This occurs when explanations are dissociated 
from process data both temporally and spatially. Some of the characteristics of 
classic explanation that impede a cooperative interaction in a diagnostic, 
evolving situation are its retrospective, one-shot nature, its context- 
independence and its expert-to-novice relationship. The cognitive 
implications are that the explanation occurs as an interruption to on-going 
lines of thought, increases workload by requiring the practitioner to read and 
understand an explanation at a busy time, to engage in interface management 
tasks and to integrate the system's information with process data. 

In contrast, interpretations and explanations among team members 
en g a g e d in dynamic fault management are not given in one long chunk from 
one agent (problem solver) to another (problem holder). Team members 
en g a g e i n joint interpretations in which both are involved in a process of 
coming to a mutually held interpretation. The strategy among human team 
members is to maintain a common ground as the situation evolves. They 
assist one another in maintaining an up-to-date interpretation in several 
ways. For example, they draw attention to anomalies, events and parameters 
of concern and they speak about them relative to expectations. Team 
members also provide informative responses, i.e., with elaborations tailored 
to the information needs in the current context. Communication among 
human team members, like conversation in general, reflects a sensitivity to 
what is informative and relevant to others (Grice, 1975). Team members also 
provide unprompted communication of relevant activities (i.e., their 
influences on the process) and assessments. They talk about strategies and 
evaluate the effects of past interventions. These communications provide a 
context in which information takes on meaning. This articulation of 
strategies and expectations among team members has been noted in a 
simulator study of coordinated activity in aircraft flight crews (Orasanu, 1990) 
and investigation of coordination between airline operatios center and central 
flow control in air traffic management (Smith, Orasanu, McCoy et al., 1994). 
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In both cases, the investigators find that coordinated activity requires 
investment to build a shared model for the situation and the perspectives of 
the different agents. Similar results have been obtained in other studies of 
coordinated activity across people (e.g., Hutchins, 1990) and in particular in 
studies of cockpit resource management (e.g., Foushee and Manos, 1981). 

A result of the role of the common ground in communications is that 
it is unusual for team members to ask "why do you say that?" or "why did 
you do that?" These questions, which express a need for explanations, 
indicate a rift in the common ground. Breakdowns in cooperative interaction 
between pilots and cockpit automation are marked by just these questions: 
what is the automation doing? why is it doing that? what will it do next? 
(Wiener, 1989). These breakdowns in cooperation between people and 
automated systems have been linked to weak feedback about the current and 
anticipated activities of the automated system (Sarter and Woods, 1995) Such 
"strong but silent" systems do not function as team players (Woods, in press 
a). Similarly, studies of human interaction with intelligent system indicate 
the need for a common frame of reference to support true cooperation 
between the human practitioners and AI advisory system (e.g., Roth et al., 
1987). 

Why Invest in the Common Ground? 

In general, team members invest heavily in communicating about the 
state of the monitored process and the state of the problem solving process. 
There are several good reasons for them to make this investment in the 
common ground. One is that diagnosis entails disentangling the various 
influences acting on the monitored process, some of which may be due to the 
interventions of other team members (human and machine agents). Hence, it 
is important for team members to assist in keeping one another aware of 
their interventions on the process. 

At another level, an important reason to invest in the common 
ground is to help keep other team members in a state of readiness so they are 
able to assist in the management and diagnosis of faults in the process. The 
same level of effort to keep someone updated is not warranted if they are not 
true team members. This is reflected in an episode found in one case in 
which an update to a medical student was cut short in order to deal what was 
perceived to be a more pressing task. 

Another important function of maintaining a common ground is that 
it can allow for more efficient communication during higher tempo periods; 
less needs to be said because information can be communicated relative to 
what is already mutually known. 1 This is consistent with Orasanu's (1990) 
findings concerning the temporal-sensitive nature of communication among 


^or example, grounding allows references to the same item to become more concise during the evolution of a 
communication task (Clark & Wilkes-Gibbs, 1990; Krauss and Weinheimer, 1964). 
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cockpit crew members; she found that captains in high performing crews 
talked less than captains in low performing crews when workload was high- 
also, the captains of high performing crews requested slightly less 
information during abnormal phases of flight, whereas captains of poor 
performing crews requested more information during these phases 

Establishing common ground can make the need for retrospective 
explanations of assessments or actions less necessary. This is useful because 
such explanations would be resource-consuming at high-tempo, high- 
criticality times, when concentration needs to be devoted to understanding 
the process behavior, rather than in mending a problem in a team member's 
understanding (recall the episode in which the attending puts off an 
explanation of his decision until a more opportune time). In this purpose 
maintaining common ground is similar to anesthesiologists' preparatory or 
anticipatory behaviors (Cook et al., 1991; Xiao, 1994), i.e., a task undertaken at 

the moment, so that things will be easier later on, when they can be expected 
to be more busy. r 


Implications for Human-IS Cooperative Interaction 

Much research on artificial intelligence explanation assumes that 
explanation is linguistic. Swartout and Moore (1993) state that for an expert 
system to generate good explanations, it must meet the desideratum of 
linguistic competence, i.e., "it must be able to construct a coherent 
multisentential text to achieve a communicative goal." This aspect of more 
sophisticated explanations is still in the research stage. The danger of 
intelligent systems for supporting dynamic fault management that 
incorporate a limited degree of linguistic competence is that they would end 
up being "chatty" and distracting. 

The study presented herein supports the notion that distributed 
dynamic fault management relies on maintaining a common ground. We 
su 8g es t that this is a conclusion at a competence level of analysis; that is, all 
team players engaged in dynamic fault management, whether human or 

machine need to invest in the common ground. How this is to be done is a 
separate issue. 2 

Even though the metaphor of a "conversational system" is problematic 
tor dynamic fault management, there are certain lessons that may be drawn 
from human-human communication. This is basically Brennan's (1990) point 
when she says that direct manipulation interfaces succeed because they share 
important features with conversation. Brennan points out that general 
strategies from human-human conversation apply to human-computer 
interaction design; these include: provide feedback (akin to the 
"backchannels" of conversation), have ways of establishing that 


2 
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take this action, or what "Why don't you put the A-line in" means, why it 
would need to be done and how to do it. 

Another context is shared local knowledge, that is, shared knowledge 
about how the team, or particular team members, tend to do things that can 
be done in more than one way. Often there multiple "correct" ways to do 
something, and the department or team may have particular ways of doing 
them, e.g., the default induction drug to use. Also, team members may have 
different "styles." For example, the data showed attendings varied in their 
approach to drug dosage or fluid replacement therapy; these variations are 
stable and are recognized by other team members 4 . 

Another context is the shared temporal context. This refers to 
knowledge about the history of the process, including what interventions 
were taken, what the evolution of the state of the process has been and of 
problem solving. A brief statement like "pressure's 100" in an update gains 
its significance (i.e., is this expected, normal, should we do anything to 
intervene?) depending on factors established in the past course of the case. 
These factors include: whether the patient is a chronic hypertensive, whether 
certain drugs have been given, whether certain events have occurred or are 
about to occur (i.e., an aneurysm clipping typically requires that hypotension 
be induced immediately prior to the clipping). The shared temporal context 
(coupled with shared domain knowledge) allows such a statement to be 
understood qualitatively— as a state, e.g., either low or high, depending on the 
mutual knowledge of the case. 

A fourth context is the physical context which consists of both the task 
environment and the set of available monitored process representations. 
Communicating within the context of the same physical environment means 
that grounding is less costly because the constraints of copresence, visibility, 
audibility, and cotemporality are present (see Chapter 2). These constraints 
allow team members to ground without explicit informing; information is 
available about what other team members do through peripheral access- 
being able to see what others do, even though one is not explicitly monitoring 
for it. The other aspect of the physical context concerns the monitored process 
views. The transcripts showed that team members often talk about 
interpretations of the process while looking at displays and pointing. Pointing 
(deitic reference) makes for compact communication —pointing to some item 
on the display can substitute for a description or an explanation in some 
situations. Certain representations can provide a wealth of other information 
e.g., analogical, trend information, that can be had "for free" when using 
deitic reference. 

Aeents vs. Tools 


4 In case 2, the 


senior resident tells the junior resident "[All likes to fill them up, [A2] likes to keep them dry." 
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The need to maintain a common ground means that a "dark board" 
strategy, in which the intelligent system draws attention (i.e., sounds alarms) 
only when "something is wrong" is inappropriate model for communication 
between human practitioners and intelligent systems (Woods, in press b). 
What, then, would be an effective approach for conveying the IS's relevant 
assessments and actions on the process? How can common ground be 
maintained between human and intelligent system? 

People import expectations from human-human communication in 
their interactions with machines (Suchman, 1987). Because of the opacity and 
pseudo-animacy of such systems--they seem to take actions of their own 
accord 5 human partners may be wont to take an intentional stance towards 
them and to imbue them with more intelligence than they deserve (Woods, 
in press a). Norman (1990) points out that current automation has an 
intermediate level of intelligence; it is smart enough to take actions and offer 
assessments, but not smart enough to act to handle all abnormalities and to 
provide the continual, appropriate feedback found among human operators. 
Norman provides an example of an autopilot that fails to provide feedback 
that it has reached the limit of its compensatory ability. He points out that for 
a system to be able to inform team members about this state of affairs, it 
would require a "higher-level of awareness, a monitoring of its own 
monitoring capabilities." To be able to do this in the general case, requires a 
degree of intelligence that is not yet been attained in the research labs. 

Rather than developing the agent-like or stand-alone properties of 
machines, another approach for creating joint human-machine cognitive 
systems is to design them as tools to support practitioners in their field of 
activity (Roth, Bennett and Woods, 1987; Woods, 1993). In this view, 
information from the AI system is another form of data to be integrated with 
other raw data in supporting situation assessment. In this approach, 
understanding of the AI system's activities and assessments is supported, not 
by more sophisticated linguistic explanations, but by making functioning 
apparent. 6 


In the Context of the Monitored Process 

The critical desiderata for diagnostic support is that it be "in synch" 
with the tempo of activity, efficient and not distracting. An important aspect 
of this is that assessments be integrated in the context of the monitored 
process views. Intelligent system assessments and activities that are 
dissociated from the process views can lead to extra cognitive tasks (Potter, 
and Woods, 1991; Remington and Shafto, 1990). Studies indicate that, when 


5 The aircraft flight management system (FMS) responds to operator inputs as well as to situational and 
system fartore. For example, the FMS initiates a mode transition when a preprogrammed intermediate target is 
attained (Sarter and Woods, 1993). 5 

This is related to Suchman's (1987) point about an artifact being self-explanatory in two senses; 1) it can 
explain itself as a human might do when queried, or 2) its functioning or use can be easily discoverable. 



66 


communication demands with the machine agent are high, particularly 
during high tempo periods, practitioners will abandon cooperative strategies 
and switch to single-agent strategies (Woods, Johannesen, Cook and Sarter, 
1994). Continuous display of AI system reasoning, if dissociated from the 
monitored process, is still likely to be distracting because it would draw the 
operator's attention away from the process. 

Integrating IS assessments into the process views isn't simply about 
spatial contiguity; it means making the basis on which the diagnostic 
information is generated, apparent. One source of information that should be 
made apparent in the representation is what process data is being used (e.g., 
Roth, Butterworth and Loftus, 1985). Another important source of 
information used by model-based intelligent systems is the context-sensitive 
expected values generated for critical parameters. While this model-based 
capability is used for diagnosis, it could also be used to provide a context for 
data presentation to the human team members --so that team members know 
what referents, expectations, and predictions are being used in the IS's 
assessments. Expectations are important to convey because they set up the 
contrast cases for explanations. A common frame of reference is supported if 
these expectations are made apparent. 

Common Frame of Reference 

The common frame of reference concept arises from work in 
distributed (multi-agent) problem solving that indicates breakdowns occur 
when multiple agents (some of which may be machines) engaged in problem 
solving do not have access to the state of the problem or the problem solving 
approach taken by other collaborating agents (Roth, Bennett and Woods, 1987; 
Hutchins, 1990; Suchman, 1987). The common frame of reference is about the 
resources that allow for a common and accurate understanding of the state of 
the process and state of problem solving across team members. 

As mentioned earlier, there are typically many sources of data about 
the process, some of which may be processed by an "intelligent agent." When 
there is no common frame of reference, data will be available piecemeal, 
relevant relationships will not be emphasized, and data will be divorced from 
its context. It can lead team members to form potentially diverging and 
inaccurate ideas about the state of the process and of the problem solving 
process. It can also lead to increased cognitive workload because practitioners 
are required to integrate various sources of data. Also, when the costs 
involved in coordination with a decision/problem solving support system 
are too high, practitioners have been known to abandon cooperative 
strategies, abandon use of the support system or constrain its use in severe 
ways (Woods, 1993a, Remington and Shafto, 1991; Cook and Woods, 1995). 

The common frame of reference concept expresses the need in effective 
distributed problem solving of integrating information into a single 
framework that highlights relationships among the data and places data in 
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the context of assessments of current and expected states of the process. Figure 
16 attempts to illustrate the common frame of reference idea. 

Effective joint problem solving requires resources that allow for a 
common accessibility of the problem state and of the problem solving 
approach (Roth, Bennett and Woods, 1987). What this means for dynamic 
fault management is that it is important to have: 1) accessibility to the 
problem solving approach-- the capability to know what hypotheses have 
been considered and rejected, to know the relevant assessments and activities 
of others, and 2) accessibility to relevant information about the monitored 
process-- the capability to become aware of the relevant information at the 
right time concerning the process. Relevant information means information 
that is tailored to the interests and expectations of the observer. 


now 



An important aspect of a common frame of reference is that it supports 
economical attention-directing reference or "joint reference." (Woods, in 
press b). This relies on there being some external representation of the 
conditions and events in the referent that is available to all agents. A shared 
external representation allows for "shared mindset across the cooperating 
agents about the background field against which the agents can all recognize 
interesting conditions or behaviors." (Woods, in press b). 

Integrating IS and process views entails designing effective monitored 
process views that can be well coordinated with the IS's assessments and 
information about its actions. Potter et al. (1994) indicate how a function- 
based display can be designed to use the IS's computational power to help the 
operator visualize the behavior of the monitored process. The function-based 
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display integrates the results of the IS's computations into the display of the 
monitored process state, and so creates a shared frame of reference (Woods, 
and Roth, 1988). Another representation that creates a shared frame of 
reference is an event-driven timeline display (Potter and Woods, 1991). This 
representation spatially segregates messages in terms of monitored process 
events and intelligent system assessments, while highlighting relationships 
among the events and the intelligent system assessments and placing them in 
the context of a timeline, so that temporal relationships are apparent. 

Integrating IS assessments and activities into the context of the 
monitored process is a promising way, given the capabilities of current 
technology, to avoid retrospective explanations. In this way, when the IS 
presents assessments, they make sense to the human supervisor because of 
the previous context, rather than appearing "out of the blue," as a surprise to 
be investigated. 

Future Directions for Research 

This research has been a first step towards understanding the nature of 
team member support in dynamic fault management and the role of 
explanation. To further understanding of how team members establish 
common ground, it would be useful to refine the conditions under which 
they tell one another about assessments and activities. For example, in 
updates it would be useful to be able to predict what parameters team 
members will mention and how they will inform others about them given 
the case's history. We can form some preliminary hypotheses in this regard, 
based on some of the findings from the study. For example, one hypothesis is 
that updaters will call attention to certain parameters that have become 
anomalous, that have continued to be anomalous and parameters that have 
gone from anomalous to normal during the team member's absence. 

A factor to investigate more deeply is the relationship between the 
nature of team member assistance and team member roles and relationships. 
What patterns of diagnostic support are found, for example, among NASA 
flight controllers in mission control? The roles and relationships among 
team members are different from those among the attending and the 
residents, in mission control there are several flight controllers, each with a 
highly specialized area of expertise, who support a flight director (supervisor) 
in making high level decisions. 

The findings of this study indicate several ways in which team 
members maintain a common ground. Another important issue to 
investigate is how team members detect gaps and and repair "rifts" in the 
common ground. A simulator study in which particular scenarios can be 
created may offer a useful approach at this stage. 
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APPENDIX 
LIST OF CASES 


— 

CASE 

TYPE 

LENGTH 

RESIDENTS 


1 

Laminectomy 

10 hrs. 

1 

— 

2 

Cerebral aneurysm 
clipping 

5 hrs 

2 


3 

Brain tumor 

5 hrs 

1 


4 

Laminectomy 

4 1/2 hrs 

2 


5 

Laminectomy 

4 hrs 

2 


6 

Laminectomy 

3 hrs 

2 

— 

7 

Cerebral aneurysm 
clipping 

6 hrs 

2 


8 

Cerebral wound 
infection 

5 1/2 hrs 

1 


9 

Laminectomy 

2 3/4 hrs 

2 


10 

Laminectomy 

8 hrs 

2 
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